aadilqau,
This is a common question and has been asked other places on the forum. It is important to understand that the BER is fundamentally random as it depends on random noise and even the random bit sequence created by the Pseudo-Random Sequence Generator (ok not truly random, but for all intents and purposes!).
Thus even from simulation to simulation without changing any parameters the BER can change drastically. This is where the sequence length comes in. For short sequences the BER can vary quite a bit depending on the exact sequence. Having a larger sequence length helps to average out this random behavior and give a better approximate BER (check out: https://en.wikipedia.org/wiki/Expected_value).
Samples per bit is easier to understand and its impact. Since on computers we need to approximate the simulation as a discrete problem the time variable is discretized. The continuous nature of time is sampled and approximated. Since a lot of the models in OptiSystem depend on the time value, a larger samples per bit value means the simulated discrete system is a better representation of the real physical system. In other words a larger samples per bit value will increase the accuracy of the simulation at the cost of computing time.
Regards