Have I forgot, my only Love, to love thee, Severed at last by Time's all-severing wave? (3-4)
We don't usually think about forgetting to love, but the speaker makes it clear here that even love is susceptible to memory and time. The thought of forgetting to love makes the speaker feel a little guilty, as if time has severed her faithfulness in remembrance.
Faithful, indeed, is the spirit that remembers After such years of change and suffering! (11-12)
We hear more about the speaker's anxiety over her faithfulness in these lines, as she tries to convince us that a spirit that remembers is indeed faithful, especially after so many years. Faithfulness was mighty important to the Victorian audience, so it makes sense that we'd hear the speaker voicing her concerns in that regard.
And, even yet, I dare not let it languish, Dare not indulge in memory's rapturous pain; (29-30)
We don't normally think of memory as something indulgent, but it certainly is here. It needs to be checked when it involves such "rapturous pain." See, when you wallow, memory can do you more harm than good. That doesn't mean you have to forget. According to our speaker, you just can't let memory consume you.