The poem has two settings: in the speaker's head—where visions of Love, Ambition, and Poesy dance—and a lovely, lazy summer morning, which the speaker spends either in bed or under a tree.
As the poem opens, the speaker is just waking up. Three figures appear to him, representing Love, Ambition, and Poesy (poetry). We can assume that these figures appear in his head, rather than as ghosts.
Regardless of whether they exist in reality or just in his mind, the speaker is annoyed by their presence. Why? Because they interrupted his lazy summer morning, which he goes on to describe with flourish as a "blissful cloud of summer indolence."
His summer day is so nice, in fact, that it is no match for the figures that tempt him. The poem may begin inside his mind, but it ends with a quiet nap in "flowery grass." This quiet, idyllic landscape matches the one in his head—at least, when the figures aren't around.