The speaker now envisions himself (or at least, his "representative") as living in a world where the last of religious faith has died away and the church has been deserted.
The speaker admits that future-him would be bored by the church, and would be generally uninformed about what it's supposed to stand for.
(By the by, we hope that you're picking up here that the speaker is really describing himself when he talks about his representative. Since he's wondering about the future, he just needs to add this wrinkle to, you know, cover his time-travel plot tracks. These descriptions, though, should be seen as personal reflections.)
The speaker knows something about "ghostly silt." Should we be impressed? Well, that depends on what he means. Silt is a kind of dirt really. More specifically, it usually means dirt that has been left behind by a river or flood. For a personal representative, wandering around run-down buildings in the far-off future, the concept of being left behind seems pretty appropriate here.
More than that, this left-behind dirt is "ghostly." With that description, the speaker plays on the double meaning of the word ghost, which can mean either the presence of something dead, or the living "Holy Ghost" that forms part of the Christian Trinity (along with God and Jesus). The duality of the word "ghostly" actually sums up the entire emotion of this poem, since we can't be sure if it falls on the belief side of religion (Holy Ghost) or the disbelief side (i.e., "For cryin' out loud. There's no such thing as ghosts and even if there were, they're just the pale imitation of living beings").
Dispersed, yet tending to this cross of ground Through suburb scrub because it held unspilt
We learn more about this ground, the literal dirt that the speaker's future representative has come to. It looks like it has dispersed over time. So long, silt. We hardly knew ye.
Still, that doesn't seem to bother our future-speaker. What's he up to? Why, he's "tending"—caring for, or paying attention to—the ground.
It's not just any old ground, either, but a "cross of ground." The speaker's rep. seems drawn to the religious importance of the place on which he stands.
We know he likes this place because he has to reach it through "suburb scrub" (48). That sounds like it could cause a rash.
More specifically, the mention of the suburbs here (where all the houses are alike, neatly organized in a place that is usually far-removed from the diversity—and energy—of the city) puts us in a world of typically predictable isolation. The suburbs often symbolize a lack of emotional connection between people, who decide to move out of the cities in order to avoid being in close contact with one another. However you might feel about the 'burbs, the "scrub" (weeds and such) that the speaker mentions seem to indicate that he's not a fan.
So what's he doing there? What's so great about an old, forgotten chunk of dirt, anyway? Well, we learn that "it held unspilt." Even though the soil is scattered now, it seems that there was a time when this religious ground was bound together. In this way, the speaker's appreciation indicates the symbolic nature of the ground itself. It's not just a bunch of dirt he's tending to. He's dealing with the church itself—both as a physical building and as a spiritual concept.
So long and equably what since is found Only in separation—marriage, and birth,
The speaker goes on to say that he looks after the ground because it (read: religion) has managed to hold together certain things that, in the modern world, are only known by separation. Marriage, for example, is an institution of the church, but in the increasingly secular modern world, divorce rates are much higher than ever before (fun fact: Larkin at one point carried on a committed relationship with three women at once).
By the same token, birth once meant a person was brought into a community of religious worship. Now though, it doesn't even necessarily mean that a person is brought into a coherent family unit.
And death, and thoughts of these—for which was built This special shell? For, though I've no idea
Also, the church is able to make sense of life and death, giving significance to this event—as it does to birth and marriage—by talking about the immortal human soul and its eventual journey into heaven. This experience is what is contained in the "special shell" of the church that the speaker mentions.
Now, though, Larkin's speaker gets only a sense of separation in death (as with birth and marriage). Importantly, this separation is also the result of "thoughts of these" things, too. Not only do these major life events not mean anything in this church-less future, but the very act of thinking about them is likewise fruitless, producing only separation. Sheesh. Somebody pass the happy pills, please.
Now, let's talk about that question mark, shall we? If you scroll up, you'll find the start to that question, way back in line 45. That line ("Or will he be my representative") is essentially the whole question in a nutshell. Since then, remember, we've been getting details on what that future representative is up to: being bored but tending the cross of ground. Good times.
What this accoutred frowsty barn is worth, It pleases me to stand in silence here;
In the final lines of stanza 6, the speaker is having some real estate issues. How much is a "frowsty barn" worth? Search us. If we had to guess, though, a barn filled with Frosties would be pretty valuable, not to mention delicious.
What's that? It's frowsty, not Frosties? Aha. Well, nevermind. A musty barn can't hope to match one filled with delicious ice cream treats. At least it's "accoutered" (furnished), though. Still, that doesn't help our speaker assign a value to the place.
By calling the church a "barn," he once again shows that for him, the place is just a really big building with no greater significance.
Still, he seems happy to hang out there, not saying anything. After all his criticizing, he can't escape the appeal that the church has for him.