Start with B2B scheme (without any Transmission fiber), if it got better then the problem with the fiber parameters like nonlinearities and dispersion. if there was no improvement in BER in B2B then remove (minimize) the system noise or change the power level!
hope that helps,
thanks
Mohamed Abouseif