Hi Maha,
It is quite easy provided that first you go through the OptiSystem tutorials about how to co-simulate the software with Matlab, whose files have been attached here by Damian and you can also easily find the videos. But then you can create the filter directly without having to use a matlab component as one block in your filter.
Regards