JOEL: If only I could meet someone new. I guess my chances of that happening are somewhat diminished, seeing that I'm incapable of making eye contact with a woman I don't know.
It's not that Joel doesn't like people, and it's not that Joel doesn't have an exciting life. It's just that he's scared. Scared of rejection? Scared of opening up? Whatever he's afraid of, it has something in particular to do with connecting to members of the opposite sex.
Joel experiences memories of himself in the Lacuna office while Howard explains the procedure to him and Stan shows him the items he brought in.
As this scene plays, what really conveys Joel's fear and confusion and anxiety to the viewer is the music, which you can listen to here. When picks up pace at the end, that signifies the point at which Joel's memories become a montage, and he flashes quickly through multiple memories at once that don't quite make sense. In the present, Patrick is bumping around with the equipment; Joel sees a red light and sporadically hears voices coming from the room, which only adds to the chaos in his mind.
JOEL: I wish I'd stayed. I do.
CLEMENTINE: Well, I came back downstairs, and you were gone.
JOEL: I walked out. I walked out the door.
JOEL: I don't know. I felt like a scared little kid. I was like... It was above my head. I don't know.
CLEMENTINE: You were scared?
JOEL: Yeah. Thought you knew that about me.
Joel isn't someone who likes adventure; he's more the stay-at home-with-an audiobook-and-a-glass-of-deionized-water kind of guy. So when Clem starts pulling him into a situation in which he is uncomfortable, he doesn't know what to do except escape. (Okay, to be fair, Clem and Joel did break into someone's house, and Clem was about to wear these people's clothes and drink their wine, so it wasn't exactly your typical night on the beach. But still.)
JOEL: Uh, don't go too far. […] I think I should go back.
CLEMENTINE: Come on. Come on!
JOEL: What if it breaks?
CLEMENTINE: "What if"? Do you really care right now?
The ice is a metaphor for intimacy. Joel is afraid to take a leap of faith and put his trust in someone. If that someone breaks his heart, he will be left drowning in sorrow and regret… Or maybe it's just ice, and Joel's scared of bruising his bottom. Who knows?
JOEL: She's not looking at me. She's busy. She's not looking at me. No one ever looks at me. I want my mommy. […] Don't leave me, Clem. Oh my God, Clem.
CLEMENTINE: This is sort of warped.
JOEL: I'm scared. I want my mommy!
Baby Joel just wants to be noticed, like most babies or toddlers or whatever he is. Maybe Joel was actually neglected as a child, which might explain some of his insecurities. But most likely, he was just having a moment of fright when his mom went briefly out of sight.