When there's a lot of talk about eyeballs in a book called Invisible Man, you know something's up with sight. Reverend Barbee gives a crowd-pleasing speech praising the Founder of the college only to later reveal that he is a blind man. Then Brother Jack turns out to have a false left eye. This shows the flawed nature of their visions – Barbee gave a great speech praising an institution and man that are basically shams, and Jack espouses a horribly cold ideology.
As for the narrator, he comes to believe himself an invisible man because no one actually sees him for who he is – but as someone of whom they can take advantage. Realizing this social invisibility, the narrator decides to pair it with actual invisibility, and drops out of sight for an indeterminate amount of time.