We really don't get to meet the speaker in this poem. In fact, the only person present in the poem is Mariana, with whom our speaker seems very familiar.
The speaker sets the scene immediately. It's dark, gloomy, and run-down at the farmhouse, and things aren't looking too good for ol' Mariana. He (we're just assuming he's a he) knows her moods, and can hear her nightly refrain.
But does the speaker seem at all concerned? Nope.
Think of the speaker like the narrator of a play: he sets the scene, tells us what we need to know, and then sets the scene again the next day.
There's one question we can ask here, though: why doesn't Mariana get to narrate? Well, she herself is a character in a play, and they usually don't get to narrate the scenes while they appear in them.
Ultimately, we don't really get a speaker in a conventional poetic sense. We get more of a dramatic narrator. That seems appropriate for this poem, though. By setting up the poem similarly to the way a play would be set, Tennyson seems to be giving a nod to his main inspiration for this: Shakespeare.