As i am in office right now. but i would like to ask what parameters u have set for the layout window. try setting sequence length 1024 and samples per bit 64. Also reduce the bit rate for each channel to 10Gbps. How many channels are there in your designed mux? try solving with the given parameters.
Thanks
Sanjeev