Usually, the question of a poem's speaker is a pretty straightforward one. You simply locate the character who is narrating the action and—bingo—you have your speaker.
Initially, it seems that simple in "The Blessed Damozel." We have someone who is describing the actions of the damozel, the architecture of the house of God, and the astronomical arrangements of heaven. In that way, our speaker seems to be a third person omniscient presence, knowing and seeing everything that's going on in the poem.
Then we come to stanza 4. It's written in parentheses, which indicates that it's an aside, a kind of incidental note separate from the poem. That's all well and good, but it's the way that the note is written that should give you pause. In these lines, the speaker is no longer the third-person, all-seeing speaker from the previous three stanzas. He's actually the damsel's beloved, wondering if the damsel's "hair/ Fell all about my face…" (21-22).
Specifically, the use of first person in these lines clues us in to a separate, second speaker. This is the guy for whom the damsel pines. And, as it turns out, the feeling's mutual. He comes back a few times (in stanzas 4, 11, and 17) to express his own sadness at being apart from his dead lover.
The damsel, though, never gets to use first person. Instead, it's the first omniscient speaker who tells us how she feels (hopeful, confident, nervous, or sad) and what she's up to (leaning over a balcony, mostly). This goes on until the two speakers actually meet up at the very end of the poem. We hear from the living lover (still speaking in first person, still using parentheses) when he confirms what the omniscient speaker has already described for us: "(I saw her smile)" (139).
This is kind of an odd moment. We have our original speaker telling us that the damozel smiled, then we have our second speaker telling us the exact same thing. So what's the difference? Well, the lover should not really be able to see or hear the damsel. She's up in heaven, remember, while he's stuck down on Earth.
All the same, he's been fantasizing throughout the poem about seeing, or hearing, his long lost lady. This crossover in the poem's last stanza, then, suggests that his connection to her is so strong, it's as if he's becoming as powerful as an all-seeking speaker.
The two speakers in this poem, then, do more than just narrate the poem's story. They allow Rossetti to explore the power of love, which can turn a poor, Earth-bound lover into an all-seeing, all-knowing poetic speaker. Good for him.