Curiously enough, the word "Babel" doesn't appear anywhere in the story. Which means it must be another example of Borges' use of "intertextuality." Intertextuality occurs when one text or narrative comes up within another narrative in a way that shapes the meaning of the larger text. It's a technique Borges uses all the time – for another example, check out our lengthy discussion of this story's epigraph.
The title is a Biblical reference to a story in the book of Genesis, which explains the origin of different languages. In the story, a unified and monolingual (they all speak a single language) human race sets out to build an enormous tower in the city of Babel that would reach up to the heavens. (Check out Pieter Brueghel the Elder's vision of what the tower might have looked like.) God wasn't such a big fan of this idea, so he caused human beings to speak many different languages so they could never come together to work on a project that ambitious again. The word "babble," meaning to utter incoherent sounds or to talk excessively, has its root in this Biblical story.
Borges' Library is an enormous, monolithic structure full of books written in languages that no one understands. See the connection?