The speaker is hanging out in a churchyard just after the sun goes down. It's dark and a bit spooky. He looks at the dimly lit gravestones, but none of the grave markers are all that impressive—most of the people buried here are poor folks from the village, so their tombstones are just simple, roughly carved stones.
The speaker starts to imagine the kinds of lives these dead guys probably led. Then he shakes his finger at the reader, and tells us not to get all snobby about the rough monuments these dead guys have on their tombs, since, really, it doesn't matter what kind of a tomb you have when you're dead, anyway. And guys, the speaker reminds us, we're all going to die someday.
But that gets the speaker thinking about his own inevitable death, and he gets a little freaked out. He imagines that someday in the future, some random guy (a "kindred spirit") might pass through this same graveyard, just as he was doing today. And that guy might see the speaker's tombstone, and ask a local villager about it. And then he imagines what the villager might say about him.
At the end, he imagines that the villager points out the epitaph engraved on the tombstone, and invites the passerby to read it for himself. So basically, Thomas Gray writes his own epitaph at the end of this poem.