Science Fiction, Philosophical Literature, Literary Fiction, Postmodernism (?)
One of the reasons we love Borges so much is that he combines the fun of science fiction with the depth of philosophical literature. Okay, so this story is a little light on the "science" part of science fiction, but because it depicts a universe based on mathematical and logical concepts, we think it counts.
The question of whether Borges is a postmodern writer is a hotly debated one. On the one hand, this story does have a few qualities shared by a lot of postmodern literature – mainly its "self-referentiality" (the fact that it recognizes itself to be a work of literature) and its "intertextuality" (the fact that it recognizes the existence of other works of literature). Additionally, it was published in 1941, smack dab in the middle of World War II, which is widely acknowledged to be the beginning of the postmodern era. On the other hand, this story doesn't have any of the darkness or irony associated with the horrors of WWII and the rise of postmodern literature.
Borges is known for helping to form the genre of Latin American magical realism, but this particular story isn't a very good example of that, as it doesn't really contain any magic. In fact, this story is sort of the opposite of magical, since the Library is based on reason, logic, and order. For a better example of Borges' use of magical realism, check out the stories "The Aleph" and "The Lottery in Babylon."