The Phantom Tollbooth is all about freedom. Milo is free to travel and have imaginative adventures, so long as he has the tollbooth to take him there. So it's strange, then, that he keeps getting himself trapped in sticky situations. In fact, Milo has to rescue himself and many other characters and things along the way, including the sounds imprisoned in the Soundkeeper's fortress, and Rhyme and Reason, who are trapped in the Castle in the Air. Plus, we can't forget the harrowing adventure that ensues when his entire posse gets trapped in the Island of Conclusions. You'll notice that whenever these people or things get trapped, the key to escape seems to be tucked away in their own minds somewhere. All you need to do is keep that mind open, and see where your wits lead.
When King Azaz and the Mathemagician banished Rhyme and Reason from the Kingdom of Wisdom, they really confined themselves and their people to a world without wisdom.
Perhaps Rhyme and Reason had the power to return to Wisdom at any time, but didn't want to do so until the people they had left behind showed their true appreciation for what had been lost.