"A stately pleasure dome decree" (line 2)
This dome is the most important symbol of civilization in the poem. It is the image of safety, peace, and order. The speaker returns to this image throughout the poem. We don't learn much about what it looks like, but we know that domes are symmetrical and strong, and that they shelter us from the weather. In other words, the dome is everything that the chasm, the river, the caverns, and the ocean are not.
"Through caverns measureless to man" (line 4)
We could have picked any number of natural spaces out of this poem, but this one seems like the most important, and the one that returns the most often. The caverns are a symbol of everything in nature that we can't understand or dominate. These caverns are dark, mysterious, and full of secrets. The speaker emphasizes all these points, and underlines the difference between man and nature, when he says the caverns are "measureless to man." They are beyond our reckoning; they cannot be controlled by our science.
"Through wood and dale the sacred river ran" (line 26)
Again, this is just one of many examples of nature in this poem. At this point the river is quiet and harmonious. It isn't throwing up boulders or crashing into the sea. Even though it is outside the dome, it is called the "sacred river." It is part of nature, but humans have a relationship with it. There's a long history of people creating a relationship with nature by worshipping it. Here the idea recalls the Greeks, who had sacred relationships with rivers, streams, and springs.