First and foremost, this poem sounds like a song if we ever heard one—and not just any song, but a kid's song, sort of. Think about it; it rhymes every other line pretty regularly, the lines are short (only three beats), and the poem itself is short and repetitive. This means it's easy to memorize, like lots of kid's songs. Try humming the first stanza to yourself to get an idea:
So we'll go no more a roving
So late into the night
Though the heart be still as loving
And the moon be still as bright.
Still don't believe us? Check out this overly somber version.
There's more to say about this poem than "it sounds like a song," however. This is a poem about not roving, right? Yes it is. Even though the speaker is resolved to stop roving, he still makes the poem sound like roving; this is how he gets his fix, without actually going roving. It's the equivalent of virtual reality: you get to do something without really doing it. And the poem's sounds help to get that idea across.
For example, you will notice that in the poem's final stanza, the words "roving" and "loving" occur, just as they did in the first stanza. We are transported back to the poem's beginning when we hear these words, only with a slight difference (in the first stanza "roving" comes first; in the last, "loving" comes first). The repetition of these words, and the poem's repetitive rhyme scheme (ABAB CDCD AEAE), give the poem a sense of (sonic) purpose: the speaker's resolution not to rove is matched by a poem whose sounds don't rove.
What we mean is, the poem keeps coming back to the same sounds, rather than straying all over the place and trying out different rhymes and sounds and that sort of thing (note, for example, how many words in this poem contain the letter O). When we encounter the words "loving" and "roving" in the poem's final stanza, we feel that we have returned from a short, circular journey. We arrive at the same place from where we set out; we rove, but not really, since we come back. The poem has a destination, so to speak, while roving, by definition, is wandering with no particular destination. And just as the speaker talks about this through the poem's content, the sounds at work here are preoccupied by the same challenge: do I stay or do I go?