Diegetic sounds are sounds that occur "in-world": things like dialogue, gunfire, passing street noises, and anything else the characters themselves might hear. Non-diegetic sounds are things like the music of the soundtrack and voice-over narratives, which the characters presumably can't hear.
From the beginning, Hitchcock wanted to keep non-diegetic sounds to an absolute minimum in Rear Window. Franz Waxman's score gets a little opening flourish but then disappears until the closing credits. The rest of the time, we hear only those sounds that actually occur in the film's world. Even more, we only hear them the way Jeff hears them. Conversations in other apartments are heavily muted, for instance, while we hear cars and other noises the way Jeff would in his apartment. This technique keeps us tied to Jeff's point of view.
Just as we're only seeing what he sees, we're only hearing what he hears.