HELLO DHIMAN..
Well i had once come across same problem and i had posted that on the forum too.. i myself was confused what did happen but i want to tell you that i had increased the bandwidth of MUX and DEMUX as alistu had suggested..Try checking the filter order of the mux and demux.. Also see if your symbol rate is greater than the bit rate in the global parameters..I hope u get it solved.
Thanks & regards