Lots of places get mentioned, alluded to, conjured up, and hinted at in this short poem. Sometimes, it's more of a question of what isn't in here rather than what is. Some of the places mentioned are real, others not so much. The interplay between them, however, is what really matters.
Let's start with the latter, the made up locales, since they're more fun. The poem is stocked with mythological references, and every time the speaker drops one of those bombs we think of a particular place. Thus, when he says the name Helen, we think of Troy and the huge battles that took place there during the Trojan War (for more on this myth, see our "Detailed Summary"). Now, even though this place is semi-mythical, it still reminds us of the very real Mediterranean climate and vegetation, alluded to when the speaker talks about that "perfumed sea" (3).
The same pattern holds in the poem's second stanza. The speaker talks about Helen's "hyacinth hair" (7) and "Naiad airs" (8). Both remind us of mythological beings and places (an athletic event with gods in it, and strange forests haunted by river deities, respectively), but they also remind us of ancient Greece, the very real historical location. And, as if he could read our minds, the speaker mentions Greece (and Rome, for good measure) a few lines later. We momentarily have an image in our heads of the Roman coliseum and the Greek Parthenon (what else would be synonymous with "glory" and "grandeur"?).
The various settings of "To Helen," and the interplay between them, complement perfectly the speaker's treatment of Helen. On the one hand, she's a real woman—a real person who can stand in the window of a real house. On the other hand, she's practically a goddess, a woman straight out of some mythological fantasy, so beautiful she can't possibly be real! As we jump back in forth between our real and mythological settings, then, we're actually going back and forth along with the speaker in his alternating treatments of Helen. Pretty neat, huh? Now, who said settings weren't important?! Location, location, location…