"Disillusionment of Ten O'Clock" was written in 1915, so middle-class America was still holding on to the hopes that it could be the European middle class. It was all a bit stuffy, and Stevens was in the thick of it. But this poem isn't about repression like your classic, sexless Victorian Novel was. Instead it calls for the reader to be like the old sailor, sound his or her barbaric yawp, and dream of hunting tigers. Let loose, in other words. At least in your dreams, if nowhere else.
Lines 3-11 build up a sense of mental containment and confinement that can only be overcome by imagining new possibilities.
Stevens uses "haunted" to signify the intangible and unnoticed absences of excitement and color. It's not obvious, but it's totally there.