Well, there are brief jaunts elsewhere: walks to town, walks around the lawn, and the all-important lighthouse journey, but the house is where it’s at. Such a specific locale creates a particular enclosure around the Ramsay family and their assorted friends and colleagues, allowing Woolf to run in and out of their minds at will and create a true sense of background for each of them, as well as a sense of unity during the dinner party scene.
Most of the novel’s action – let’s make that "action" – takes place before and after World War I, which places the characters in a pretty specific time frame, the main repercussion being that gender is a big deal. Unmarried woman who likes to paint? Dime a dozen these days, but not so much back then.