Dear Ekrem,
You may right click on the BER Analyzer component to access the properties window. Then select Help to access the datasheet, where you will find are theoretical information on the process used to calculate the BER and Q-factor.
Regards,
Ahmad