They left the high-road, by a well-remembered lane, and soon approached a mansion of dull red brick, with a little weathercock-surmounted cupola, on the roof, and a bell hanging in it. It was a large house, but one of broken fortunes; for the spacious offices were little used, their walls were damp and mossy, their windows broken, and their gates decayed. […] There was an earthy savour in the air, a chilly bareness in the place, which associated itself somehow with too much getting up by candle-light, and not too much to eat.
They went, the Ghost and Scrooge, across the hall, to a door at the back of the house. It opened before them, and disclosed a long, bare, melancholy room, made barer still by lines of plain deal forms and desks. At one of these a lonely boy was reading near a feeble fire; and Scrooge sat down upon a form, and wept to see his poor forgotten self as he used to be. (2.53-54)
It's interesting that none of the places where we see Scrooge is actually a home. They are all either temporary housing or in some other way they lack just about all the qualities that make a house a home—safety, comfort, and love. Is this why he doesn't have a problem spying on other people with the ghosts? Because he has never really experienced the safe, cozy feeling of a real home?