Right off the bat, we can tell there's something funky going on with memory in this poem. We think "immemorial" is an interesting word to use here. In general, when we say immemorial, we mean that something has been going on so long that we can't remember when it started (e.g. "Kids have loved candy corn since time immemorial"). That doesn't really make sense in this context, though. Maybe Poe is sort of reinventing this word, since that's definitely his style. It could mean something more like "unforgettable" or, alternately, "impossible to remember." In either case, Poe puts the theme of memory front and center here.
Our memories were treacherous and sere— (line 22)
Poe's poems and stories are all about altered states of mind. His main characters are often imbalanced in some way (crazy, drunk, depressed, grief-stricken, etc.). Here our speaker hints that there was something not quite right upstairs when he took this walk. He says his memory, like the leaves on the trees, was "sere" (dead, brown, lifeless). He also calls it "treacherous," which makes it sound like a person, just waiting to stab him in the back. In this poem, memory isn't just a fact of life, it's a threat to your sanity.
We remembered not the dank tarn of Auber, (line 28)
Forgetting is a big part of this poem. It's what creates the mystery that makes this poem exciting. If our speaker remembered everything, there would be no suspense, no surprise. He and Psyche have forgotten where they are, even though they have good reason to remember. He never says why they forgot, but we suspect the terrible grief over losing Ulalume has forced him to forget.
To the Lethean peace of the skies— (line 46)
Lethe is a Greek word meaning "forgetfulness" or "oblivion." In Greek mythology, when people died, they would drink from the river Lethe, which would make them forget about their past lives. We think it's pretty significant that, when the speaker dreams about heaven, he doesn't imagine music or angels or happiness. He just thinks of it as a place of forgetting. This seems like another clue that he's in pretty bad mental shape. If you were feeling healthy and balanced, you probably wouldn't fantasize about forgetting everything you ever knew.