(Remember, this is a memory that we're hearing about from the Underground Man.)
When the Underground Man gets to Simonov's place, there are two other men there, both guys that he knows from school. They basically pretend to ignore him. He figures they hate him.
The men are busy talking about a farewell dinner they're having the next day for an old schoolmate who is going away. The man in question is Zverkov, an officer that the Underground Man, not surprisingly, despises.
Actually, he does justify his hate. Apparently Zverkov is handsome, playful, and rich, which makes everyone want to get on his good side. As such, he's also "swaggering" and a master of "social graces."
This part is particularly detestable to the Underground Man; Zverkov owns an estate with 200 serfs. (Remember, this was back in the days of the feudal system, when a man owned massive amounts of land and rented out pieces of it to farmers, who would give a large percentage of their yield to the lord of the estate – Zverkov, in this case.)
Zverkov was a womanizer. He used to brag about how he would have sex with all the new brides of the serfs who lived on his land, because that was his right as the landlord.
Other men used to applaud him for these aspirations, but the Underground Man resented him.
Well, you're thinking, that sounds reasonable!
Not for long. The Underground Man explains that he doesn't feel compassion toward the new brides. Instead, he just refuses to applaud an insect like Zverkov for any reason.
Anyway, the Underground Man and Zverkov have been less than friendly toward each other for a while.
With that background info out of the way, the Underground Man tells us some more about the two other men visiting Simonov. One is named Ferfitchkin, a "German blockhead" and a bitter enemy of the Underground Man's. He sucks up to Zverkov because he knows he can borrow money from him.
The second man is named Trudolyubov, who is concerned only with things like social status and getting a promotion at work. He thinks of the Underground Man as useless (at least, that's what the Underground Man tells us).
So it is these two men, along with Simonov, who are standing around making their plans for this going-away dinner when the Underground Man interrupts and assumes that he's going to the dinner as well, adding that he would help to pay by contributing seven roubles, just like everyone else.
The men are taken aback at his blatant self-invitation. Simonov, awkwardly, says something like, "Oh…did you want to come too?"
The Underground Man responds with something like, "I can't believe you weren't going to invite me!", and the men quite reasonably ask why they would invite someone who hates Zverkov to Zverkov's going-away dinner.
The Underground Man responds that he wishes to come precisely because he has never gotten along with Zverkov. He makes it out as though he wants to reconcile with the man.
Trudolyubov sighs and gives in, leaving only Ferfitchkin to complain about how the Underground Man shouldn't be allowed to come.
And on that note, the two visitors exit, leaving the Underground Man alone with Simonov.
Simonov walks around the room, flushed and awkward, and drops a hint reminding the Underground Man that he (the Underground Man) has owed him fifteen roubles for ages. The Underground Man starts to apologize but Simonov waves its off and continues stomping about the room.
Finally, the Underground Man asks, "Am I keeping you?", and Simonov, seeing his opportunity for a quick exit, promptly says "yesiree Bob," or something to that effect.
So the Underground Man promptly gathers his coat and hat and dashes for the door, surprised at himself for his own socially acceptable response of leaving when asked.
On the way home, however, he gets horribly angry at himself for asking to tag along to this dinner.
Still, he tells us in retrospect, what made him the most angry was that he knew he would attend the dinner even though he wasn't wanted.
And there's another problem: he has no money. He has nine roubles, but he has to give seven to his servant, Apollon (plus he still owes Simonov fifteen, but we guess he's not even thinking about that yet).
He decides simply to not pay Apollon and to use the seven roubles to pay for his share of the dinner.
That night, he has horrible dreams, mostly having to do with memories from school. We get a glimpse into his personal history here, as he reveals that he was sent away to the school (sounds like a boarding school) by the "distant relations" who had custody of him.
At school, he explains, his peers used to taunt him because he was different from them. He couldn't stand their jeers, so he hid himself away and hated them in secret.
Still, he knew that he was above them; their stupid minds couldn't possibly comprehend "the essential things" of life, as his own could. Worse still, they all interpreted rank as an indication of intelligence. And all this because they were stupid.
While he hated them all, though, he admits that he was "perhaps […] worse than any of them."
Then he took the attitude of, "FINE! I don't like you anyway. Hate me all you want" and simply focused on his schoolwork, where he excelled.
When teachers started to notice, the students stopped mocking him, but they were still hostile.
The Underground Man couldn't hold out in his cynicism forever; he ended up longing for companionship.
At one point, he says, he did have a friend…but then he acted as a "tyrant" toward him. The friend, seemingly a weak and passive fellow, "devoted himself entirely" to the Underground Man, at which point the Underground Man started hating him. And that was that.
Back to the Zverkov dinner story. The Underground Man wakes up the next morning (the day of the dinner) as excited as a kid on Christmas morning. He feels that today, miraculously, his life will change.
But he also realizes that (much like the Nevsky incident) there are a thousand details to take care of: polish his boots, examine his clothes, etc. He's incredibly nervous for the evening and wonders how Zverkov will act when he shows up. He imagines that Zverkov will treat him disdainfully. He's particularly nervous that his poor appearance – including the stain on his pants – will shame him.
Worst of all, he thinks, the evening will likely be "paltry and unliterary." (This goes back to his earlier discussion of the sphere of art. These men, he thinks, are beneath him and can't understand his lofty, educated thoughts.)
And yet, despite knowing this, he can't help but dream that the other three men – Simonov, Trudolyubov, and Ferfitchkin – will be taken with his (the Underground Man's) charm and abandon Zverkov.
The Underground Man realizes, though – and realized even then, he says – that the most humiliating part of the whole fiasco was the fact that he didn't even care if he humiliated Zverkov. He didn't even care if he won over the friends and took them away from his enemy. Frankly, dears, he just didn't care.
And still, when the clock strikes five, he heads out for dinner, sneaking past Apollon, who still hasn't been paid his wages.