When we start talking about how the poem sounds, there's something we can't ignore: there's some major repetition going on here. Just as the speaker keeps turning back to his obsession with forgiveness, Donne turns back to these trusty words and phrases. The first two stanzas repeat "Wilt thou" twice, and both end with the refrain "When thou hast done, thou hast not done, / For I have more."
Donne doesn't just repeat phrases. The word "done" itself is repeated seven times, "sin" is repeated six times, and "more" is repeated three times. That might not seem like much, but for a poem with only eighteen lines, it certainly is noticeable. What, did this guy not own a thesaurus? Likely, it isn't a limited vocab that's causing all this repetition. This is a deliberate technique. But why, you ask?
Well, keep your poetry pants on, Shmoopers, and we'll tell you. For one thing, Donne does call this poem a "hymn." (Check out "What's Up With the Title?" for more on that.) All this repetition, then, gives the poem a musical predictability, a rhythm. That's totally appropriate, since hymns are, you know, meant to be sung. Repetition is also the sonic hallmark of obsession. To put it more directly, our speaker is really hung up on all this sin business. To his thinking, it's impossible to get away from it. He keeps coming back to the same words and phrases, then, as echoes of his anxiety, reminding us readers of his obsessive state of mind.