By the way, I have just check my previous guess with my old version of OptiSystem. I have noticed that each single time (by default) the simulation runs a random value is used for noise and the bit sequence generator (PRBS). Hence, theses random settings should make the resulting BER vary from one run to another! However, if you start increasing bit sequence length the difference between simulations decreases and you receive similar results for all your runs.