Pale grew thy cheek and cold, Colder thy kiss; (5-6)
Cold, colder, and pale—sounds like a dead body to us. Sure, this girl isn't really dying, but for all intents and purposes she'll be dead to the speaker for a while, if not forever.
The dew of the morning Sunk chill on my brow—(9-10)
The dawn is usually a symbol of rebirth, but here it's totally dark and ominous. Dew is nice, but the "chill" here is no good. It recalls the coldness of the woman's cheek and kiss in the first stanza and thus again makes us think of death. Brrr.
Thy vows are all broken, And light is thy fame; (13-14)
Broken vows equals dead vows. There are no ifs, ands, or buts about it. The speaker's old flame has definitely "killed" something (a marriage or a relationship) that they were supposed to keep alive together.
They name thee before me, A knell to mine ear; (17-18)
"Knell" is synonymous with death. Always. Period. It is the bell rung to announce somebody's death. The mention of the woman's name is like the announcement that somebody is dead—is it the speaker, who feels dead or feels like death? Is it the woman, who is essentially "dead" to the speaker? It's hard to tell just whose death is being heralded by the "knell" here.
In silence I grieve, That thy heart could forget, Thy spirit deceive (26-28)
Like we said, death is sneaky. Here he arrives in the form of forgetfulness. The speaker might as well be dead to the woman whose "heart" has entirely forgotten about him.