I'm a bit late for the party: I've been postponing familiarizing myself with Classical Japanese and therefore avoiding all content concerning it, including this thread. Last week I finally started reading this thread and bought a book on Classical Japanese by the Russian linguist N.A. Syromyatnikov (his book on Old Japanese was also on sale, but I decided to get that one a bit later).
On the very first page of the book proper (i.e. excluding the intros), in the segment about the vowel system, the author discusses the avoidance of
hiatus in Old Japanese by merging vowels at the borders of words and/or morphemes, and how it at least partially carried over into Classical Japanese. He actually cites 字余り as an example of this phenomenon: while the written representations of the poems keep the morphemes on those borders intact, to make the text more comprehensible, the original readings would omit one of the clashing vowels. Therefore, even though the mora count of the text looks excessive (thus also making it excessive in modern readings), the original readings of the poems still fit the standard 5-7-5-7-7 rhythm. The example he gives is the line 水の面に, transcribed "midu-no omo-ni", but originally pronounced "midu-nomo-ni". He cites L.M. Yermakova's research of "Yamato-monogatari" as the source for this, while also adding his own conjecture that the omissions might not have happened any more if the speaker made even the slightest pause between the two words (thus explaining lines like はつつる糸(いと)は), and that it's likely that they were no longer obligatory in spoken Middle Japanese and were preserved in poetry as a tradition.
A cursory glance at the poems in this thread reveals that the instances of 字余り do seem to be limited to lines containing orthographical hiatus. In response to Zgarbas' question, it might be that it's not the あい(あひ) and なん(なむ) that are considered a single mora, but the りあ and もあ. However, considering that the first edition of this book was published in the USSR in 1983 (the second and current one only came out in 2002, long after Syromyatnikov's death), I wonder if this theory has any currency in modern Japanese studies anywhere outside Russia. I have already noticed other bits and pieces of dated and/or non-mainstream info in other parts of the book. For instance, while the author doesn't go into detail about the theories of the origins and classification of Japanese, from time to time he somewhat matter-of-factly drops references to proposed Altaic cognates and grammatical parallels in Austronesian languages, implying that he accepted both the Altaic theory and the Austronesian creole theory, the former of which is highly disputed and the latter IIRC no longer considered credible by most scholars.
So, yudantaiteki, I guess what I would like to ask is whether you have heard anything about this explanation for 字余り, or about any evidence that could disprove it (like a poem from the early Classical period where 字余り occurs in a line without any hints of hiatus).