Smooth rhythms and loads of similar sounds make this poem as soothing on the ear as an oozy crème-filled chocolate egg. Alliteration and assonance give sound spotlights to important lines while internal rhymes and visual assonance keep the focus on form as well as content.
You can hardly read two words of this poem without smacking up against alliteration or assonance. Suddenly everything either begins with the same sound or contains the same vowels. In addition to making the poem sonically striking (translation: it sounds cool when you read it aloud) these two stylistic devices serve as tasteful red flags, grabbing your attention and directing it toward a particular line's meaning. It's no surprise then that the two arguably most important lines in the poem, 10 and 20, are chock full of both alliteration and assonance (alliteration is bolded, assonance is bolded and italicized):
Then shall the fall further the flight in me (10)
Affliction shall advance the flight in me (20)
Notice how your ear lingers over all those F's and A's? That's exactly what Herbert wants. These two lines summarize the basic paradox of the whole poem (and notice that they both say the same thing but in slightly different words): despair brings joy. What the what? It's a variation on the same paradox behind Easter itself, which celebrates how, according to Christianity, Christ died so that everyone else could live again.
But's it not just the heavy-hitting lines that get styled up. Herbert scatters alliteration and assonance with a liberal hand. In line 11, for instance, we've got a short I assonance in "did" and "begin," and in line 12 Herbert seriously goes to town with a specific type of alliteration known as sibilance, which just means lots of S's:
And still with sicknesses and shame (12)
If a poem hits you with that many hisses, yeah, you're going to remember it.
Internal rhymes (words that rhyme within a single line rather than at the ends of different lines) and visual assonance (words that contain the same vowel but actually sound different) are important in a different way. Less noticeable than alliteration or regular assonance, they stand out visually, drawing your eye in the way alliterating words catch your ear. And in shaped poetry, where layout plays a big role, visual devices are really important to overall meaning.
Let's look at the first line. Notice two words that look alike but sound different?:
Lord, who createdst man in wealth and store (1)
Yep, the EA in "createdst" and "wealth" visually binds them together, even though they're pronounced totally differently. So what's the point? Well, chances are, if your eye links "createdst" and "wealth," then your brain will too, and that's exactly the point this line is making. God created Adam's wealth.
Something similar happens in line 8, where an internal rhyme on AR links "larks" and "harmoniously." Although you can hear internal rhymes, they're pretty subtle and easy to miss among the more obvious assonance and alliteration in the poem. What's more important is this pair's visual connection, emphasizing the link between birds and song.