Hello
Please make sure that the signal structure is correctly defined in the Matlab code. It must match the input to the next component in OptiSystem. There are different types (sampled, parametric, noise bins, etc..). Please refer to the example in OptiSystem Example library located at the following link for more guidance.
C:\Users\USER NAME\Documents\OptiSystem 17.0 Samples\Software interworking\MATLAB co-simulation
Regards,
Ahmad