This is definitely a poem about how saying goodbye was really hard and painful for the speaker, and how being separated from his old flame is just awful. If the poem is about saying goodbye and feeling lonely, it's also about how these experiences break or sunder many other things: hearts, vows, etc. In other words, goodbyes create all kinds of ruptures, fractures, and the like. And what are ruptures and fractures but separations?
Lines 1-2: The speaker and the woman part with "silence and tears." Silence will continue to be a big part of the poem. It is also a symbol of the speaker and the woman's impending separation (people who are apart don't talk and are thus "silent" toward one another).
Lines 3-4: Images of separation abound here. The half-broken hearts symbolize the pair's sadness as well as their breakup or "severance."
Line 13: Not only have the speaker and the woman broken up, she, apparently, has broken some "vows" she made to him. The "vows" that once united them, in other words, no longer hold.
Lines 17-18: "Knell" here is a metaphor for how the woman's name sounds to the speaker. It sounds like the bell that is rung to announce somebody's death. So, the woman's name even reminds him of the separation that is death (somebody who dies is no longer "with us").
Lines 21-24: The speaker imagines an end to what has been a lengthy separation. Ironically, the potential reunion of the lovers will resemble their departure: it will be quiet and tearful, at least on the speaker's part. It sounds like he will be eternally separated from his old flame, even if he "meets" her again.