An owl-eyed man at a Gatsby party sits in awe in the library, murmuring with amazement that all the books on Gatsby's shelves are "real books":
"See!" he cried triumphantly. "It's a bona-fide piece of printed matter. It fooled me. This fella's a regular Belasco. It's a triumph. What thoroughness! What realism! Knew when to stop, too - didn't cut the pages. But what do you want? What do you expect?" (3.50)
Quick Brain Snack: books used to come with their pages uncut, meaning that the sheets that are folded to make the books aren't sliced open on the top. You'd have to cut them open before reading. If you didn't, everyone would know that you hadn't actually read the book.
Gatsby's uncut books tell us that much of what Gatsby presents to the world is a façade. He wants people to believe that he's a well-educated man, an Oxford man, but in fact he only spent a short time there after the war. He wants people to think that he's well-read, but he's never even cracked the covers. So, the simple answer is that the books represent the fact that Gatsby is a fraud. He's built up an image of himself that isn't consistent with the facts of his life. But you could also argue that the unopened, unread books represent Gatsby himself: eternally mysterious, eternally unopened.
Well, maybe not eternally mysterious. We think we've got some ideas: check out his "Character Analysis" for a peek inside those flashy covers.