Okay, so the monster didn't ask for all this. But does that really exonerate him? Last we checked, no one was getting off death row with the defense of "I can't be guilty of murder; I didn't ask to be born." So when you remember that the guy was in some way responsible for four deaths or so (five if you count Victor), then he stops looking like a sympathetic character. Not to mention, if you went with Victor as the protagonist, the monster, his enemy, is the antagonist by default.
If you went with the monster as a protagonist, then Victor has to be the antagonist (or possibly all of mankind, everywhere. Except for the blind people). Victor antagonizes the monster by refusing to have compassion for him, by refusing to support him after creating him, and by refusing to make him a companion. Lots of refusal going on here.
The really sexy analyses claim that both Victor and the monster are antagonists not only to each other, but also to themselves—that they're just two sides of the same being. "Ooooh," you say. Yes, "Ooooh" indeed. And check this out: what do we usually call the monster?