Hi lavanya, I am giving you some theoretical concept why the ISI occurs in WDM system, You may find this helpful.
one of the reason for ISI in wavelength division multiplexing is due to the less channel spacing, i.e. less difference in wavelengths or frequencies and at a high data rate transmission. On way to reduce the intersymbol interference (ISI) is to use return-to-zero (RZ) format in place of Non Return to Zero (NRZ) format. The RZ format in optical communications has advantages over the more frequently used non-return-to zero format, mainly because the RZ-modulated signal can withstand better the impact of fiber nonlinearity and polarization-mode dispersion. the most widely used method is the polarization interleaving method. However, for the very small channel spacing, the power leakage from one channel to its adjacent channels still remains. To reduce this leakage, one need to use filters with sharp spectral response, typically consisting of more than one stage- such filters are expensive, exhibit high insertion loss, and often cause large intersymbol interference (ISI). the adjacent channel interference (ACI) can be reduced by the dispersion interleaving method. these are the two methods- POLARIZATION INTERLEAVED & DISPERSION INTERLEAVED WDM SYSTEMS. you may find so many papers on these two topics on the internet.Once you make the design you have to see which method will be fruitful for you.
Regards
.
.