Where It All Goes Down
There's one thing we know pretty much right away about the setting of the poem: at least in the speaker's imagination, there's a rainbow. Perhaps it's just rained, the air feels fresh and cool, and the earth is still damp. Imagine the colors up in the sky: red, orange, yellow, green, blue, indigo, violet (we still remember ROYGBIV from our childhood, too).
Whatever the case, the setting is important to this poem in a very general way. It's not that our speaker is just a rainbow fanatic. Sure, he thinks rainbows are swell and all, but that's not the only thing we think he's on about here.
Really, the rainbow in this poem is a symbol of the speaker's appreciation for the natural world as a whole. It's a part of nature, but by appreciating it specifically, the speaker is really telling us about his love for all things in nature. That's why we learn in the last line of his "natural piety." He's not just telling us about his rainbow t-shirt collection. He's communicating his love of the whole of the natural world.
It may seem like a broad and useless thing to say that this poem is set "in nature," but really that's the whole key to understanding what Wordsworth's up to here. Nature is the star of this show, and the speaker's relationship with the natural world is what this poem is all about.