"What gives you the right to address me this way?" he said, though he didn't really feel the anger his voice carried. He was about to add, Are you my wife?—but choked those words off. To his perpetual shame, he and Almaz had been intimate twice over the years, both times when he was drunk. She'd lain down, lifted, and spread, grumbling even as her hips fell into rhythm with his, but no more than she grumbled about the coffee or hot water. (2.11.21)
Sex is basically shameful for Ghosh, because he's always doing it with women he doesn't love. But here it's especially shameful because with Almaz, he has abused his power as her employer and has taken advantage of her. Think about it—what would be the consequences of her saying no? Possibly losing her job and her income.