Nobody heard him, the dead man, (1)
It's been said that everybody dies alone, and this line hammers the fact home. Not only is he alone in death, but he's apparently cursed with the ability to keep speaking, even though no one can hear him. That's gotta be torture.
I was much further out than you thought (3)
If he's swimming, then he's too far from shore or the other swimmers. We have to wonder why he's so far out. Was he being reckless? Did he overestimate his swimming ability or mistake the water's depth? Maybe a kiddie pool would have been more his speed.
Oh, no no no, it was too cold always (9)
The phrase "out in the cold" can mean "excluded." So being out where the water is too cold seems like a metaphor for being cut off from the warmth of home and companions. The dead man felt he was always excluded from these basic comforts.