The ‘other world’ here is a metaphorical one, and consists largely of Sebastian’s appreciation for beauty. It all begins with the trip to the botanical gardens. Of course, Brideshead Castle plays a large role in constituting this ‘other world’ as well.
As a burgeoning artist, it makes sense that Charles would be so taken in by the splendor of Brideshead Castle. He spends pages describing its design, architecture, and furnishings.
The perfect world starts to crumble when Charles realizes the extent to which religion and family torment Sebastian. He chooses to side with his friend, which means making a temporary enemy of Lady Marchmain. Samgrass frustrates matters further, especially since he imposes restrictions for the boys even at Oxford.
Charles has become a part of the world of Brideshead Castle, but it isn’t all sunshine and rainbows. He senses a darker side to himself correlating to what he discovered as the darker element of this ‘other world.’
Charles ‘escapes’ the world of Brideshead and returns to reality, leaving the Flyte family behind him completely. The interpretation of this conclusion as an ‘escape’ is certainly subject to debate, since he was really more evicted than anything else. It’s also subject to debate whether or not leaving was a positive thing for Charles, or an unhappy bit of tragedy.