"Casey at the Bat" sounds like what it is: a ballad. But there are also some subtler sound aspects at work in "Casey…" that we'll explore here. (If you want to know more about that ballad sound, check the "Form and Meter" section.)
In addition to all those strong rhymes and regular meter that make this poem a ballad, Thayer also makes good use of alliteration in some key lines. Take a look at line 18:
It rumbled through the valley, it rattled in the dell; (18)
Hear those repeated consonant sounds, those R's and D's? That's a kind of alliteration called consonance. In this context, the repeated sounds mirror the rumbling and rattling that the line is describing. It doesn't hurt that those words are also pretty onomatopoeic on their own.
Alliteration is also a good way to inject some extra musicality and rhythm into lines. That way those strong end rhymes don't get all the attention.
Here's another example from line 34:
Like the beating of the storm-waves on a stern and distant shore.
In this example, we can hear how those repeated, hard T sounds echo the "beating" that the line describes, while the repeated S sounds hiss along in a snake-y, ominous way.
Thayer takes repetition a step further in the poem's last two stanzas. The phrase "and now" is repeated three times in stanza twelve's last two lines (that's called anaphora), and the word "somewhere" is repeated five times in the poem's final stanza. This kind of repetition really helps to build suspense as the poem nears the end and we wonder whether Casey will win the game or not. It kind of sounds like the "and then," "and then," "and then" of an excited kid telling a story.