"Alone" is a retrospective poem, which means that it's a poem told by a guy looking back on his childhood. This complicates the whole speaker issue. The guy who is actually speaking in the poem is, of course, an older, more mature version of the guy he's describing. However, the speaker is also that younger child that he describes in the poem. It's almost like he temporarily transports himself back in time and reassumes his former identity.
So let's talk about the younger version of the speaker a little bit, because that's who dominates the poem. Now, this isn't a poem about bullying or getting made fun of, but the kid in the poem feels completely alone and isolated. His tastes, passions, and even his sorrows are completely different from everybody else's.
The speaker of this poem isn't just some lonely guy, however. He's also special. He's alone, sure, but because of that he gets to experience a kind of "mystery." We don't know exactly what this is—it is a mystery, after all—but we get the feeling that it's not entirely a bad experience. The speaker associates this mystery with powerful, inspiring views, ones that only he can see. That makes things seem just a little better now, doesn't it?
The last thing we have to tell you is that this poem is very autobiographical, which means it is one of many places where Edgar Allan Poe talks about himself, reflecting Poe's own sense of his difference. He was orphaned at a young age (his father took off before he was born and his mother died when he was very young), and he generally felt out of place. "Alone" very openly describes the young Edgar Allan Poe, and his own feelings of both isolation and inspiration.