hi AASif …
in one of the post in optiwave it was discussed by alistu .Hope it will help you..
Power penalty means when a parameter in the system changes and results in degradation of the eye-opening, the amount of power that compensates for the degradation is called “power penalty”. For this, you would have to do the simulation before the change in that certain parameter (implementation1), and then after that change in that parameter (implementation2). Then you should increase the power in implementation2 to the point where eye diagram becomes the same as implementation1. The value of the power you have added is the power penalty.
For your reference i am attaching the link.
https://www.google.co.in/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&cad=rja&uact=8&ved=0ahUKEwjPqZeBjLTLAhXCj44KHWiZAYAQFggnMAA&url=http%3A%2F%2Foptiwave.com%2Fforums%2Ftopic%2Feye-opening-penality%2F&usg=AFQjCNGZaOo5sTVethkp0YUPfH5UuLrvDQ&sig2=OVldjRBuWWqQQKkJR_ALLA