To be perfectly honest, Shmoop doesn't have much evidence that the speaker of "The Man He Killed" feels guilty at all. But we're betting he does, since he seems to stumble over his words as he describes the scene. And it only gets worse from there, as he tries to justify the death with shaky reasoning and dead-end excuses. In the end, the real guilt comes from the fact that he killed a man who was just like himself. How in the world do you clear your conscience after something like that?
The speaker feels less guilty by the end of the poem, because he understands that the man tried to kill him, too. They were a lot alike, after all.
The long dashes, internal rhyme, and repeated because and foe show us just how guilty this guy feels. He can't face the truth.