The poem starts out one late afternoon under the oaks as the sun fades and the night takes over. It's a tranquil little spot, just the place for a happy couple to chill and watch the sunset. Pretty soon, though, those oaks are under several leagues of ocean water. At least, that's what the speaker would like you to imagine. Once you're there, you're lulled by the receding tides and the shifting light above you and you're in for some surprises.
The title's "Bearded Oaks" are really just a jumping off point for our speaker's imagination, and thanks to that we soon get a much broader setting than just under the old oak tree. The speaker steps outside this moment to discuss "construction" and "dim architecture" (13-14), some kind of primeval creation and evolution that made, well, pretty much every setting we can imagine. In essence, the poem's setting is achieve through the speaker's contemplation of the broader universe, a place where "history is thus undone" (28).
Humans don't have much a place in this setting, either. The speaker recalls an abandoned city, a place so quite that " Our feet once wrought the hollow street / With echo when the lamps were dead" (29-30). Now that's a decidedly un-happening place. It's cool, though, because the speaker's not going to hang out there long. Pretty soon he's "in Time," (37) "To practice for Eternity" (40). The setting, then, is fluid, fleeting, and more about achieving a mood than nailing down an actual place. At least, the speaker's not interested in any particular place without it leading him to consider every place: the universe, all of time, all of existence. Yeah, that's not going to fit on any Google map we know.