This scene is a pretty neat set-piece—and it's full of symbolism. When Elijah and Elisha cross the river Jordan with miraculous aid, the big symbol here is pretty obvious: in the same way that Moses parted the Red Sea, Elijah parts the river Jordan. But in this case, Elijah and Elisha aren't leading Israel out of Egypt or into the Promised Land or the wilderness. They're trying to lead the people out of the worship of foreign gods—which is symbolically the same thing as being enslaved to Egypt because it's a form of slavery to a system of beliefs that Elijah and Elisha consider to be full of illusions.
Before Elijah departs into heaven in a whirlwind, riding on a chariot of fire, Elisha asks him for a "double-share of his spirit." Elijah says this will be tough, but he'll succeed in doing it if Elisha sees him fully ascend. Elisha sees him go all the way up. He then takes off his old clothes, tears them up, and then puts on Elijah's prophetic mantle.
Kind of like a Baptism
For one thing, this symbolizes that Elisha is Elijah's successor, and it's also a form of baptism. By taking off his old clothes and putting on Elijah's prophetic garb, Elisha is symbolically taking off the clothes that the fallen Adam and Eve needed to wear and putting on a new garment, symbolizing his return to an un-fallen and pure state of nature. (Consider that this happens at the Jordan, where John the Baptist later baptizes Jesus—if you happen to be reading the Christian Version of the Bible.)
Returning to the other side of the Jordan, Elisha discovers he really has received a double-sized helping of Elijah's spirit, his prophetic power. He's able to perform the same miracle with God's aid: parting the Jordan and crossing over to meet the other prophets. It shows that now that Elijah's mission is finished, Elisha is the one who will, symbolically, help lead the Israelites (metaphorically) back into the Promised Land by leading them from the worship of foreign Gods back to Israel's God.
Elijah Ascends to Heaven
Elijah knows how to make an exit. He famously departs into heaven on a "chariot of fire" carried up into heaven on a whirlwind sent by God. This image has given birth to the movie Chariots of Fire—which is about Olympic runners—in addition to plenty of allusions in poetry, literature, and elsewhere. As Parliament-Funkadelic once said, "Swing down sweet chariot, stop / And let me ride."
While the narrator may have believed that Elijah quite literally flew into the sky where heaven was actually located and lived there for the rest of eternity without physically dying, the chariot of fire has since become a symbol of passion and energy at the service of a righteous cause (like in Chariots of Fire).
The British poet William Blake said, "Bring me bow of burning gold / Bring me my arrows of desire / Bring my spear, O clouds unfold / Bring me my chariot of fire." But Blake says that this is part of "mental warfare"—it's not literal, it's part of an attempt to persuade people to view reality from a more spiritual perspective. So, although the narrator may mean that Elijah flew into the sky on a literal chariot, the history of the image in culture has lent it a new, symbolic meaning.