The speaker of "To the Virgins" doesn't say anything about marriage until the second-to-last line of the poem, where he tells the virgins to "go marry" so they don't spend their whole life being single. It turns out, in fact, that the whole poem has been about marriage. Yeah, sure, it's about making the most of one's time – about gathering those metaphorical "rosebuds" while one still can – but that turns out to mean getting married, and not going totally crazy trying to cram in all that partying while you still can.
The poem stresses the importance of marriage and implies, with some subtlety, that marriage will somehow forestall or counteract the processes of aging and death.
The importance of marriage for sexual purposes is not entirely beyond Herrick's purview, if only because of his emphasis on heat ("warmer," "glorious lamp") and marriage in the same poem.