Since Coleridge is trying to sound like a person having a real conversation, he keeps his poem relatively free of anything artificial or frilly. He isn't using a rhyming form of verse, but there's some alliteration, consonance, assonance, and anaphora (check out our examples below). It all feels very steady and direct—like something a really intelligent, eighteenth- or nineteenth-century person might actually say to you.
To talk about all this, though, we have to go back even further in the way-back machine. John Milton was kind of like Elvis Presley to the later, Romantic poets, like Coleridge and Wordsworth (who were more like The Beatles—inspired by Elvis, but going in a different direction). They liked everything Milton did, but they wanted to do it differently. So, whereas Milton writes about Satan's war with God, Wordsworth writes about earthly, everyday life and Nature. But he and Coleridge both kept an important part of Milton's style: his sound. Milton also wrote in iambic pentameter—it's the meter of Paradise Lost—and critics and readers usually describe this sonic-style as being "sonorous" (meaning it has a very full, strong sound). You could say this is true for "Frost at Midnight" as well. Consider:
Heard only in the trances of the blast,
Or if the secret ministry of frost (72-73)
Hear the daDUM daDUM daDUM daDUM daDUM pattern? It feels more like a strong steady beat—more tom-toms, fewer crashing cymbals or snapping snare drums.
As well, we get assonance, as in the repetition of S sounds in lines 5-6: "that solitude, which suits/ Abstruser musings." We get the consonance of B, L, and F sounds in lines 13-14: "the thin blue flame/ Lies on my low-burnt fire" (13-14), and later alliteration with repeated beginning F sounds: "Making it a companionable form,/ Whose puny flaps and freaks" (19-20). Finally, we can see the repetition of anaphora here as well: "Sea, hill, and wood,/ This populous village! Sea, and hill, and wood" (10-11).
So, there is just a ton of sound happening in this poem. There's so much sound, actually, that some later critics and poets, like T.S. Eliot, accused Milton and his descendants (like Coleridge) of focusing too much on these elegant, polished soundworks. They wanted more startling imagery and metaphors—not just images of Nature (like those at the end of "Frost at Midnight"). They held up John Donne and Shakespeare as prime examples of what they wanted. Eliot described Milton, Wordsworth, and Coleridge as ceaselessly "eloquent"—but he wanted something more than intelligent, melodic lines.
On the other hand, you can argue that Wordsworth and Coleridge were intentionally using simple images from nature (like frost, icicles, and robins) couched within melodic lines, for a specific reason. Whereas John Donne and Andrew Marvell would sometimes use metaphors and images from science and mathematics, or more fanciful metaphors (in "The Definition of Love," Marvell talks about "Hope" vainly flapping its "Tinsel wing" and compares his love for a woman and her love for him to two parallel lines that can never meet), Wordsworth and Coleridge look directly at the world around them, observing simple parts of Nature and human life (except in more fantastic Coleridge poems like "The Rime of the Ancient Mariner.") For this reason, sound becomes key. It helps put these simple things into a powerful sonic context. All this attention to sound elevates the subject matter. Even though the poem might be covering everyday experience, its sound effects let us know that this, too, is worthy of a sophisticated poem.