We aren't just individuals—we are part of a larger society, and John Donne won't let you forget it. Still, "The Sun Rising" is a lot about a speaker's desire to (even temporarily) escape the responsibilities and restrictions of the outside world and just experience his love. You know, without all the meddling from friends and family.
The speaker says it's not just Virginia—the whole world is for lovers. The speaker feels that lovers are above all other members of the community: workers, kings, students.
The speaker believes that the outside community (starting with the sun, but including society) will only work to spoil his love. He wishes they would all just disappear.