Donne's poems are playfully complex. He likes to layer meanings of words and create double, sometimes even triple) meanings. So, it's no shocker that sound plays a big role here, too.
A lot of that play has to do with vowels. Vowels are the breathy, sexy parts of language. Look at the first two lines. They're supposed to be light and peaceful (even though we are talking about death). Look at all the long vowel sounds ("virt-U-ous," "mIldlY," "awAY," "tO," "sOUls," "gO"). Combine that with the nice onomatopoeia of "whisper," and you've got a nice easy start to the poem. There's a similar effect of long vowels (O's and E's) with the lovely line 24: "Like gold to airy thinness beat."
Consonants get some fun in the poem, too. Usually, we see play with consonants in the form of alliteration (repeated sounds at the beginnings of words) or consonance, which is more general sound echoing even within words. Line 13 is a good example: "Dull sublunary lovers' love." All those L sounds! Those sounds are called liquid sounds—the consonant just flows right off the tongue. The parenthetical phrase right after it—"(whose soul is sense)"—combines four S sounds into four syllables, and there's more sound echoing in line 20, first with the "less" and "lips," then creating an internal rhyme between "lips" and "miss."
We should say a little something about the rhyme scheme here, but really, there's not much to say. It's a straightforward ABAB, four-line stanza (called a quatrain), where the letters represent the end rhyme for that particular line (line 1 rhymes with line 3, line 2 with line 4). The rhymes don't get passed along to the next stanza, so you aren't trying to follow it all the way down the page, and because they alternate lines, they don't draw too much attention.
There's an obvious reason for this—this is a really tough poem. It's an extremely convoluted argument with shifting metaphors and clever reversals all the way through. The last thing we need is to have to track some sneaky rhyme scheme. Even the sounds that are rhymed are pretty simple—he doesn't back himself into a corner trying to rhyme with "lasagna." Donne makes it even easier on us because he only has a couple of eye rhymes in here, words that look like they rhyme because of the letters they end with, but actually don't quite rhyme.
So what's with all the regular rhyme, assonance, and consonance? Why put so much sonic chiming in the poem, like little bells going off every few words? Well, we think the overall effect of sound in the poem makes it sound more and more like a tricky riddle or a playful tongue-twister. And if you consider its original intent—to soothe Donne's wife about their upcoming separation—then how this message gets delivered (through pleasing chimes and flourishes of sound symmetry) would have as been as important as the poem's content.