Hi Faiz, my suggestion would be to design the fiber portion using OptiSystem and then go for the Wireless mm wave communication part in MATLAB and the call inside optisystem. That would be a systematic step by step procedure. You can refer to the block diagrams as Alistu have referred in the above comment #31133. Thanking you..
Regards,
Dhiman