The master clock was anchored once to the first stream's audio PTS, but every stream starts its own PTS timeline (~90000). Subsequent streams were paced against the stale anchor, so their targets were all in the past and the video free-ran to catch up (the reported speed-up / poor fps). Reset the clock anchor when each stream starts.