Bartleby doesn't actively try to antagonize anyone – but he ends up doing it unintentionally. If we look closely at his actions, we see (like the Narrator does) that nothing he does is malicious; actually, he never really does anything to hurt anyone. Well, to be more precise, he never does anything at all. It is this very inactivity that makes him the problematic figure in this story. We usually expect negative action of some kind from our standard antagonists, and Bartleby just doesn't come through in that way. Instead, we are, like the Narrator, driven a little mad by Bartleby's mysterious nature, and it is that instinctive reaction that sets him apart from all the other characters.