Hello Lama,
please refer to the examples in OptiSystem Example Library that talks about using the DSP component for compensating dispersion and nonlinearity in the fiber. Below is a link for the example (112Gbps DP-16QAM_OSNR Control_ Transmission System with DSP.osd) that you may start to check and get more information. Please note that the fiber is unidirectional. However, you may modify the project and use bidirectional fiber. There are many things that you could explore in this example.
C:\Users\USER NAME\Documents\OptiSystem 17.0 Samples\Advanced modulation systems\QAM systems\16 QAM
Regards,
Ahmad