Pygmalion takes place in London, England in the early twentieth century. At this point, the city was the capital of the largest empire in the world. That said, we only get a very small glimpse of it. All of the play's action is confined to three places, each located in the very fashionable center of town: Covent Garden, the laboratory of Henry Higgins's apartment on 10 Wimpole Street, and the "drawing room" (think living room) of Mrs. Higgins's apartment on Chelsea embankment. You don't need to know exactly where these places are. Just know that they're ritzy, about as far away from the poor parts of London you could get.
The easiest way to understand the setting of the play is to look at who shows up where, and how they're treated. This sounds pretty easy, right? Well, that's because it is. Covent Garden was a large market on London's West End. The West End was home to many of London's theaters, and it brought together a very diverse crowd of people. Everyone from the rich (like Colonel Pickering), the middle class (the Eynsford-Hills, for instance), to the poorest of the poor – who, like Eliza, were probably trying to make some money off the richer among them. This is the only place we see a bunch of people with (pay attention to this) a bunch of accents mingling freely.
The rest of the action is confined to a lab and a living room, and in both cases the appearance of anyone "lower class" is met with surprise. In the second act, Eliza and her father can't simply walk into Higgins's place: they have to be screened by Mrs. Pearce before they can so much as get through the door. Heck, the whole play is about going undercover, about getting your name on the guest list, as it were. Eliza can only get into Mrs. Higgins's by pretending to be a lady; and Higgins can only win his bet by passing her off as a duchess at fancy parties.
Is Shaw trying to tell us that all of these high society types are full of it? That their ideas about class and sophistication are one big joke? Well, it's hard to say. He does tell Pickering that being a "lady's maid or shop assistant […] requires better English" than being a duchess; and the conversation at Mrs. Higgins's little soirée isn't exactly stimulating (1.121). Not until Eliza starts going off about her dead aunt, anyway. Still, there are plenty of indications that both Higgins and his mother aren't your average, idle rich. Everything from the scientific instruments in Higgins's lab to the art in Mrs. Higgins's drawing room speaks to their uncommon intelligence. Once again, it's hard to say exactly what this all means, but we can be sure that these aren't your typical people in typical places.