The mystical connection between humans and nature reappears again and again in early Romantic poetry. Romanticism is in large part a reaction against the idea that man is the "rational animal" whose intelligence places him above the rest of nature but also alienates him from it. At the beginning of "This Lime-Tree Bower My Prison," the speaker is indeed alienated. You could think of him like a pouting child kicking the dirt and muttering, "Stupid lime trees. What have you done for me lately?" Through the use of his imagination ("imagination" is a huge word in Romanticism), the speaker comes to realize nature's power to connect rather than separate.
The poem illustrates the idea of a single unity within the diversity of nature – a kind of Ideal Soul – but the speaker fails to integrate himself with this unity. He views it as non-human.
The speaker's reconciliation with the alienated world of the man-made garden comes at a price – he must repress his more irrational and chaotic emotions.