If we break the stanzas up into groups, say stanzas I-IV and V-IX, you might notice that the speaker in the first four stanzas has decidedly different things to say than the speaker in V-IX. In the first four stanzas, the speaker is all about dissing society and proving why nature is better; those who yearn for recognition in the public sphere are "vain" and their success is short-term and not even really that impressive (think Mckayla Maroney). Stanzas V-IX, however, shift from the rejection of other options (society, politics, and women are lame) into exalting the speaker's chosen paradise (the garden is the bomb-diggity). It's tempting to write our speaker off as a tree hugger and leave it at that, but there's another, more important element to the person behind this poem than just his or her love of nature.
Take a look at lines 51-56 to see what we mean:
Casting the body's vest aside,
My Soul into the boughs does glide:
There like a bird it sits, and sings,
Then whets, and combs its silver wings;
And, till prepar'd for longer flight,
Waves in its plumes the various light.
Here, we see Marvell use something called an epic simile (a simile that stretches on for several lines): the speaker's soul is like a bird gliding, sitting, singing, whetting, combing, and shaking its tail feathers. By comparing the activities of the soul (something people have never seen) to the activities of a bird (something very familiar), the observation suddenly becomes much more accessible and easier to understand. The bird, instead of being the focus of the passage, is a vehicle by which the speaker can make observations about an immaterial, more complex phenomenon, the soul.
The common thread in "The Garden," then, is not nature or society but the speaker's knowledge of and focus on what philosophical folks might refer to as "higher pursuits" and what we at Shmoop refer to as super-fuzzy concepts like the soul and the human mind (different than the brain, mind you). Nature, it turns out, is not our speaker's main concern; it is, instead, a magic portal through which the speaker can access and understand a higher plane of thinking. And we think that even Mckayla Maroney should find that impressive.