This is a reference to Greek language and mythology. In Greek, "Psyche" literally means "soul." The Greeks turned her into a goddess, a personification of the human soul. That's what Poe is working off of here. The speaker has separated himself from his soul, and she's able to walk and talk and think for herself.
Line 12: This is the first time we hear about Psyche. We don't know much about her yet, just that she and the speaker are walking together in a grove of cypress trees. He calls her "my Soul," giving us a little help with the mythological association.
Line 51: Here's where Psyche gets in an argument with the speaker. He's super-psyched about following the goddess he thinks he sees. She's way more cautious, and tries to slow him down. You know how you sometimes have a voice inside you that warns you not to do something dumb? We think that's basically who Psyche is, but in this case she's just popped outside the speaker's body.
Line 72: The speaker wins his argument with Psyche, calms her down, and gets her to come with him. He also kisses her. We assume that's just a comforting peck on the forehead. We're not suggesting this guy is getting it on with his own soul, which would just be weird. Still…it's just like Poe to surround his speaker with angelic, kind of sexy, ghostly women, and we think Psyche is meant to be a little bit…appealing.
Astarte was a Phoenician goddess, associated with fertility, the moon, and all that good stuff. Importantly for this poem, she was also connected to the planet Venus, which appears as the evening and morning stars.
Line 37: Here's where Astarte is actually named, although she shows up in other spots. Apparently Astarte was sometimes represented in art with a moon-like crown on her head. So when the speaker sees this mysterious glowing object coming over the horizon, he thinks of the diamond crown of the goddess. In fact, it sounds a lot like he's talking about the moon rising here. However, later references to a "star" (line 52) and a "planet" (101) make it more likely that he's talking about the planet Venus, seen here as the morning star.
Dian (short for Diana – Poe drops the second syllable to make it fit the meter) is another kind of sexy female goddess (our speaker has a one-track mind). She was the Roman goddess of the hunt and the moon. She was especially famous for being a chilly, virginal goddess who would refuse the advances of men. She was even known to kill them when they got too insistent.
Line 39: Dian just has a cameo appearance in the poem. The speaker uses her as a comparison to Astarte, who he thinks of as "warmer" and more inviting than the chilly Dian. The speaker is clearly looking for a woman who will take care of him, and he thinks Astarte is the one, even though she's technically a planet. We think it's important to notice how often women come up in this poem, especially divine and ghostly women. It's a pretty major theme, here and in other Poe works.
This is another Greek word, meaning "oblivion" or "forgetfulness." It was also the name of the river that flowed through Hades (the underworld of Greek mythology). Souls who drank from the river would forget all they had known in their past lives.
Line 46: When the speaker talks about "Lethean peace," he is imagining the comfort that will come with being able to forget his pain. He fantasizes not about being happy some day, but about some kind of afterlife where he won't have to remember his pain any more. Ugh. Poor guy.