Invisible Man is about the process of overcoming deceptions and illusions to reach truth. (One of the most important truths in the book is that the narrator is invisible to those around him.) In Invisible Man, then, deception is closely linked with invisibility. Because various people cannot see the narrator for who he is, they use him to suit their own purposes. As often as he is deceived, however, the narrator does some deceiving of his own, ultimately concluding that the ease with which he deceives people points to his invisibility.
In Invisible Man, everyone is invisible because everyone is deceiving one another as to their true identities.