Forlorn! the very word is like a bell To toil me back from thee to my sole self!
Oh, no! Why did he have to use the word "forlorn?" It reminds the speaker of how he has also been abandoned – by the nightingale itself.
All of a sudden, he gets sucked back into the normal world after several pleasant stanzas of exploring the nightingale's realm.
For him, the word "forlorn" is like, when you are having a really great dream and then all of a sudden you hear your alarm clock and remember that you have to wake up and go to class. It's a big disappointment.
The speaker is pulled back into his own mind, his "sole self."
Adieu! the fancy cannot cheat so well As she is fam'd to do, deceiving elf.
He admits that his attempts to use his imagination ("fancy") to "cheat" his way into the nightingale's world have not been as effective as he would have liked. He bids good-bye to the bird and then lashes out at his imagination for being a "deceiving elf," like the character Puck from Shakespeare's A Midsummer Night's Dream.
Although "fancy" is famed for being able to create new worlds, the speaker has not been successful at permanently escaping the everyday world.
Adieu! adieu! thy plaintive anthem fades Past the near meadows, over the still stream, Up the hill-side; and now 'tis buried deep In the next valley-glades:
Now it becomes clear that the nightingale is flying away.
The speaker bids goodbye twice more to the nightingale using the French word, "adieu," which means, "good-bye for a long time."
The bird's sad or "plaintive" song grows harder to hear, as the bird flies from the nearby meadows, across a stream, up a hill, and into the next valley. Now he can't hear it at all.
Was it a vision, or a waking dream? Fled is that music:—Do I wake or sleep?
Now that the bird is left, the speaker's not sure if he ever entered its world at all. He thinks that maybe the experience was just a "waking dream" and not really true.
But has the speaker returned to the "real" world? Maybe the nightingale's world was reality, and the "real" world is just a dream.
Everything is topsy-turvy, and he doesn't know what is true from what is fancy. He wonders if he is a awake or sleeping.