In just 32 lines, the speaker tells us a whole lot about him. We know that he had some kind of relationship with somebody that had to end for whatever reason. We also know that he was not too happy about that and is still upset. In fact, if we had to characterize this guy in one word (we're pretty sure it's a dude, though you can't safely base a poem's speaker on the poet), that word would be: sad. He grieves in silence, he says he will "rue" this mysterious person (probably a girl) very deeply for a long time. Whenever he hears her name he is reminded of death as a shudder comes over him, and he vows to confront her only with silence and tears if he ever meets her again. Yep, that sure sounds like he's still hurting from that little breakup that happened a while back, and it sure sounds like he wants to manipulate this girl a little bit, or at least get her to have a little pity party for him.
Now we said if we were to characterize him in one word, that word would be sad, but we're not going to use only one word. That would be no fun. Besides, there's a bit more to this guy than just sadness—not a whole lot more, but some. Take the poem's final stanza as an example. He says if he ever meets this girl again, he won't even talk to her. He'll be silent and cry in front of her. That's it. While that makes us think the pain will be unbearable, it also makes us think he's mad at her and wants to show her how angry he is. Snubbing somebody this way is a little aggressive, the kind of thing you do when you want to prove to somebody that you don't think they deserve to hear any words. Yikes.
So he's sad, he's a little angry, what else? How about guilty as charged? Guilty of what, you ask? Well, apparently he hasn't told any of his comrades that he was involved with this person, so that's a clue right there. Moreover, he makes that little comment about sharing in her "shame" (16), which immediately makes us think of illicit or inappropriate sexual behavior. This is the nineteenth century, after all, and secret relationships, the suggestion of biblical knowledge (i.e., sex) (21-22), and the word "shame" all point to one thing: an affair. The speaker knows this was kind of a bad thing to get involved with, so he feels a little icky about it as he writes the poem.
You know who else felt kind of icky about having inappropriate sexual relationships with (sometimes) married women? Lord Byron, the dude who wrote the poem. While this didn't stop him from having illicit relationships, he was always sorry afterwards (most of the time). Now even though Byron went to great lengths to mask the identities of the people described in this poem, it has become apparent that his feelings for one Lady Frances Wedderburn Webster (with whom he had some kind of relationship some years before he wrote this poem) were a whole lot like the speaker's. Sure, this speaker isn't really Byron: he's just some nameless guy speaking a poem. But, this speaker is Byron's mouthpiece, so to speak, a guy who sounds like Byron but who can't be positively identified with him. We'll just call him "B"—how's that?