Notice that Seymour's swim trunks are blue, while Sybil wears a yellow bathing suit. Yet Seymour says to her, "That's a fine bathing suit you have on. If there's one thing I like, it's a blue bathing suit" (2.20). If we think of blue as associated with purity or innocence, then it makes sense that Seymour is wearing blue trunks. It also makes sense that he thinks Sybil is wearing a blue bathing suit. She is pure and innocent, so he associates her with the color blue.
Actually, there's an interesting aside in Salinger's short story "Raise High the Roofbeam, Carpenters" that Buddy Glass tells about his brother Seymour. When their sister Franny was a ten months old, Seymour read her a story to stop her from fussing one night. The story he read was a Taoist tale about Duke Mu of China and an enlightened man named Po Lo. The Duke asked Po Lo to send him a man who could pick out a superior horse from a group of animals. Po Lo does, and the Duke employs this man to pick out a horse for him. The man does, and when the emperor asks about its color and sex, the man tells him it is a brown mare. When the horse arrives, however, it is a black stallion. The Duke is peeved that this guy can't even tell the color and sex of a horse, but Po Lo is ecstatic. The man has learned to look at the horse's "spiritual mechanism," he says. "In making sure of the essential, he forgets the homely details; intent on the inward qualities, he loses sight of the external."
That's seems to be what's going on here with Seymour. Seymour sees Sybil's spiritual mechanism, her internal qualities of purity and innocence. So he sees her in the color blue, rather than the yellow she actually wears.