Hello Alisthu
I have also designed long reach OFDM system from one of the optisystem samples examples,But it worked good for me upto around 4500 Km.The only thing i did is, mitigate the effects of non linearities by using matlab.I havent faced this type of problem with my system.also i have used the
the symmetric dispersion compensation with 25Km SMF followed by EDFA (Gain 5dB Nf 4dB) then DCF 10Km again EDFA and 25 KM smf.And my constellation diagrams were noisy but not showed another 4-QAM signals as you have mentioned above.