Probably the best resource to understand the calculation is the Technical Background of the BER Analyzer. Component Properties->Help.
Briefly, the component can either measure the bit sequence directly and compare, but as you said this method can take a lot of simulation time especially if the BER is low. The other main method is to use the eye diagram information to infer the BER. The algorithm takes the approximate noise of the bit value 1 level and the noise of the bit value 0 level, fitting these distributions to Gaussian probabilities. Then the probability of incorrectly detecting a 1 or 0 can be calculated as well. In this fashion, the BER of 10^-15 can be calculated from a bit sequence that is only say 1000 bits long. It is important to note though that increasing the bit sequence length allows for a more accurate result still.
Regards