"Be that word our sign in parting, bird or fiend!" I shrieked, upstarting (line 97)
There we go! All of a sudden, you're yelling ("shrieking," actually) at a bird. What's more, it's a bird that's only said one word to you. Granted, even that's weird, but screaming at birds is generally not something that sane people are inclined to do. If you were this guy's friend, this is probably the moment where you pick up the phone and call someone. He was a little edgy before, but now he seems to have tipped over the edge and seems to be truly insane.
And my soul from out that shadow that lies floating on the floor Shall be lifted – nevermore! (lines 107-8)
For us, this whole last stanza sort of seals the deal. The narrator was feeling tormented by the bird; now he feels like its shadow is keeping his soul prisoner. Frankly, that's a pretty strange thing to say. We get the feeling that he's locked in some kind of insane eternal struggle with this bird. Finally, it's a bird he doesn't know anything about. He has no reason to assume it hates him, or that is has been sent by the devil, or whatever else he might think. This prison he has built seems to come directly from his shattered mind. Over the course of this poem, he has gone from feeling a little nervous, to screaming at an animal, to becoming the eternal prisoner of a bird. While madness may be in the eye of the beholder, we think this pretty much passes the test on all possible criteria.