Three musical elements are introduced at the beginning of the song, each of which serves a different purpose: piano, drums, and flute. The piano – Apple's primary instrument – provides the backbone of the song, initially emerging in stabbing increments but ultimately guiding the evolution of the track. The percussive elements, which build throughout the song, follow a traditional hip-hop drum pattern, with prominent snare hits and up-tempo hi-hats. The flute, with its lighter quality, offsets the seriousness of the song's subject matter and highlights the singer's ironic tone, particularly at the beginning of the track (does Apple really feel bad about being a "bad, bad girl"?).
All of the instruments build in intensity as Apple transitions into the song's chorus, where additional – and somewhat cacophonous – piano parts serve to mirror the singer's inner conflict. The music comes to a crescendo in the bridge, the song's lyrical climax. As Apple pleads, "Let me know the way/Before there's hell to pay/Give me room to lay the law and let me go," the chord progression shifts; the music grows louder and more layered. In this manner, Apple utilizes sonic elements to confront her audience and enhance the forcefulness of her message. This isn't a song (or a singer) that will fade easily into the background.
Though the music plays a vital role in setting the mood for the track, the quality of Fiona Apple's voice is the song's true signature. Apple has a tattoo of the letters FHW (which stand for "Fiona Has Wings") on her back. But it would probably be more accurate to say that Fiona's voice has wings. Only a teenager when "Criminal" was recorded, Apple sounds wise beyond her years, her voice laden with a maturity and depth of experience one would expect from someone at age 39, not nineteen. Her vocal presence – commanding, mystical, and at times ethereal – is the force that ties together the song's different components and serves to cement an atmosphere of moral tension and inner strife.