Hi Abdullah,
Your filter performance is fine so far: it looks like it’s matched with your received signal and doesn’t show any visible distortions (based on your results). I don’t think that you can reduce jitter by using DCF fibers. It’s known that the main source of jitter is
improper synchronization or management of the clock scheme in the communication systems. The influence of it is increasing with the increasing bit-rate. By the way, is your speed in up-link the same as it in down-link? Did you try to increase the bit-rate and see what happens?
In general, the usage of 3R regenerators can reduce jitter but they are used for longer distances due to their high cost.