OK, just to clear things up, there are two different versions of the History House in the novel, one metaphorical and one literal.
The first is the imaginary house that Chacko uses as a metaphor for India's, and the family's, history. He explains to Estha and Rahel that they come from a long line of Anglophiles, but the story of their true family background lies somewhere else:
They were a whole family of Anglophiles. Pointed in the wrong direction, trapped outside their own history and unable to retrace their steps because their footprints had been swept away. He explained that history was like an old house at night. With all the lamps lit. And ancestors whispering inside.
"To understand history," Chacko said, "we have to go inside and listen to what they're saying. And look at the books and the pictures on the wall. And smell the smells." (2.90-91)
If you don't get that right away, don't worry – we were re-reading this quote and scratching our heads too. Let's take it one step at a time. When Chacko says the family is made up of Anglophiles, he means they are overly fascinated with all things British ("Anglo" = British, "phile" = love). By identifying with the British, they lose sight of who they really are and where they come from. To understand history – not only their family's personal history, but the broader history of which their family's story is just one small part – they have to seek out ways of learning about the past.
When Chacko says that history is like an old house at night, with all the lights on and ancestors whispering inside, he suggests that everything they need to know to understand who they are is right in front of them – they just have to look and listen carefully. The problem is, according to Chacko, that they can't go into this house:
"[…] because we've been locked out. And when we look in through the windows, all we see are shadows. And when we try and listen, all we hear is a whispering. And we cannot understand the whispering, because our minds have been invaded by a war. [...] A war that has made us adore our conquerors and despise ourselves." (2.95)
Understanding who they are and where they come from is so close to them, and yet just out of their reach. Loving England (the country that colonized India) means loving India, and perhaps even themselves, less.
Of course, Estha and Rahel don't really understand what Chacko is getting at, and it's OK if we don't either at this point. Since we're getting this moment filtered through their childhood perspective, it seems almost intentional that we should be a bit confused, too. When Chacko talks about history as a house, the twins think he's saying history is a house – a real one. And they think they know exactly which one it is:
Estha and Rahel had no doubt that the house Chacko meant was the house on the other side of the river, in the middle of the abandoned rubber estate where they had never been. Kari Saipu's house. The Black Sahib. The Englishman who had "gone native." Who spoke Malayalam and wore mundus. Ayemenem's own Kurtz. Ayemenem his private Heart of Darkness. (2.92)
If there's one most mysterious house in all of Ayemenem, it's the one across the river, in the "heart of darkness" – the darkest, most unknown and unexplored territory around as far as Estha and Rahel are concerned. (For more background on the references in this quote, check out our guide to Joseph Conrad's Heart of Darkness.) This is yet another example of Estha and Rahel's childlike version of reality; they take Chacko's metaphor literally.
Kari Saipu's house, then, becomes the second version of the History House that we encounter in the novel. It even starts to become the History House in our own eyes, since we know that when Estha and Rahel talk about going to the History House they mean going to the long-dead Kari Saipu's abode. The History House, as the site where Velutha and Ammu meet in secret as lovers, where Estha and Rahel hide after Sophie Mol drowns, and where Velutha is nearly beaten to death, becomes an important emblem of the personal history that our characters both witness and experience.