Once upon a time, everything was just hunky-dory in our speaker's life. One night, though, everything changed. Instead of awesome things like beauty and justice, the speaker embraced things like hatred and jealousy. He pulled a total 180, and not in a good way like skateboarder or BMX stunt rider. Still, the speaker can't help writing this poem to see if he can reclaim a little bit of his former mojo.
The speaker traces his lameness back to his Gaulish (ancient French) ancestors. In addition to those lazy, good-for-nothings of the past, he also hates the modern insistence on science. He just wants to get away, on a permanent vacation to an uncivilized land. He also sort of wishes he was a convict, since he identifies so much with being on the outs of society. He's incapable of loving life, but at the same time he's unwilling to embrace death. The dude has some serious problems.
Eventually, the speaker starts to fantasize that he's in H-E-double-hockeysticks (well, at least the title checks out). He describes hanging out there with his companion, who calls the speaker his "infernal Spouse" and "a demon." Ouch. Not to be outdone, the speaker reclaims the stage and starts to boast about his artistic prowess. He busts out a kind of pastoral poem as proof.
He goes on to describe the wild, often surreal, ways his imagination acts on the world, throwing in a few more formal verses to describe how he's isolated and alone. Then he returns to his fantasies of quitting civilization, or "the West," for someplace less conventional and restrained. He calls this place "the East," but he's not talking strictly in geographical terms. He wants to find a place where the restrictions of both science and religion don't apply. He wants total freedom of the mind.
As you might have guessed, such a place is hard to come by. The speaker's not going to give up, though. He looks forward to a time of rebirth, of new, "fresh," wisdom. He tries to get there through the power of his imagination, but poverty and reality combine to smack him down to the cold, dirty, stinking, not-fun-at-all Earth—bummer. He can't help hoping, though, that a new dawn is coming, where he'll be able to know truth.