Could be, depending what the encoding was, what codecs you use, and what hardware you have, it's possible those mp4s were able to decode faster, perhaps with hardware decoding or something.
Can't really say more without you trying one of the mp4s again, and giving us more information about your hardware and software.
Can't really say more without you trying one of the mp4s again, and giving us more information about your hardware and software.