Hi Sahil,
First you should clear the type of system you are designing and having problem with.
In wdm system, if you increase the power of Laser, the BER and Q-factor will increase, in addition to this if you decrease the bit rate in global parameters the BER and Q-factor may increase, some modulation formats cause less the BER and Q-factor while other give large.