Hello Arash,
Unfortunately, your project design is not correct and it will not work.
I advice you to go through the examples of OptiSystem in the “Advanced modulation systems” directory located at following link for more information on how to design your project.
C:\Users\USER NAME\Documents\OptiSystem 16.0 Samples\Advanced modulation systems\
For further discussion, please contact me at ahmad.atieh@optiwave.com
Regards,
Ahmad