Those in glass castles, even magical ones, shouldn't throw stones. While he resides in the Syrian Desert, the Jinni lives in a palace of his own creation. It's "never truly finished" and "invisible to other beings" (2.53)—which is a good way of describing the Jinni, too. The palace makes the Jinni feel a little superior, especially to the humans who can only sporadically see the castle, as though it's a mirage.
When the Jinni creates the tin ceiling for the apartment building, he embosses a map of the desert on it. Everyone thinks it's beautiful, but no one can see the palace where it's supposed to be… not even the Jinni: "I should see it. It should be there." (17.90), he says. This sad moment marks the occasion when the Jinni realizes that he can never return to his old way of life. Just as he can't see it on the map, he can't ever be part of it again.