They didn't think they could bear it if anyone found out about the Potters. Mrs. Potter was Mrs. Dursley's sister, but they hadn't met for several years; in fact, Mrs. Dursley pretended she didn't have a sister, because her sister and her good-for-nothing husband were as unDursleyish as it was possible to be. (1.3)
Mr. and Mrs. Dursley are so focused on the new family unit they've formed, and on severing any connection with the more disreputable (according to them) part of her old family, that they "pretend" such a connection never existed at all. The narrator even coins a word for it, calling Mrs. Dursley's sister and brother-in-law "unDursleyish," and really registering the specificity of how different the two families are.
"I've come to bring Harry to his aunt and uncle. They're the only family he has left now."
"You don't mean – you can't mean the people who live here?" cried Professor McGonagall, jumping to her feet and pointing at number four. "Dumbledore – you can't. I've been watching them all day. You couldn't find two people who are less like us." (1.79-80)
Right away, the text reminds us of the distinction between family you're related to and people you identify with. Harry may be related to his aunt and uncle, but they're as unlike his friends in the magic world as it's possible to be. When McGonagall doesn't believe that Dumbledore wants to leave Harry with these "people," she's indicating that they can't really be his "family," because they're so unsympathetic.
He couldn't remember his parents at all. His aunt and uncle never spoke about them, and of course he was forbidden to ask questions. There were no photographs of them in the house. (2.98)
This statement is particularly poignant once we find out about wizarding photographs, which are almost like small videos or holograms, showing their subjects animated and moving around. Harry doesn't even have stupid old Muggle pictures. His aunt and uncle try to keep him from developing any connection with his parents.
I really don't think they should let the other sort in, do you? They're just not the same, they've never been brought up to know our ways. Some of them have never even heard of Hogwarts until they get the letter, I imagine. I think they should keep it in the old wizarding families. What's your surname, anyway? (5.184)
Here, Malfoy sets up a connection between family and class. The "old wizarding families" are, to him, a higher class than the "other sort," who haven't been brought up to learn magic. Ironically, even though Harry can be thought of as practically wizarding royalty, he's in the same ignorant position as many mixed-blood wizards, and he is himself only one-generation pureblood.
Everyone expects me to do as well as the others, but if I do, it's no big deal, because they did it first. You never get anything new, either, with five brothers. I've got Bill's old robes, Charlie's old wand, and Percy's old rat. (6.150)
Having "five brothers" is a problem to Ron, because of the pressure of doing well and putting up with all the hand-me-downs, but it must seem somewhat exciting to Harry. Ron's complaining about having too much family, and Harry's never had enough.
The Sorting is a very important ceremony because, while you are here, your house will be something like your family within Hogwarts. You will have classes with the rest of your house, sleep in your house dormitory, and spend free time in your house common room. (7.7)
This statement is truer than McGonagall knows. For Harry, especially, his "house" is "like [his] family within Hogwarts"; it's also just like his family. He's practically adopted as an honorary Weasley, and he's far happier at Hogwarts than he ever was at the Dursleys'.
"I want to hear you're training hard, Potter, or I may change my mind about punishing you."
Then she suddenly smiled.
"Your father would have been proud," she said. "He was an excellent Quidditch player himself." (9.88-90)
At Hogwarts, Harry's constantly finding connections to his parents, or getting to hear people's memories of them, while back at the Dursleys' he wasn't even allowed to mention them. It's got to be gratifying for Harry to hear that he shares this special, newly discovered talent with the father he never got a chance to know.
Should Harry wake him? Something held him back – his father's cloak – he felt that this time – the first time – he wanted to use it alone. (12.103)
Harry loves being able to share experiences with his new friends, but this is different. This is the first time he's had a tangible object that connects him to one of his parents, and it's understandable that he would want to savor the experience of using it all by himself.
Harry was looking at his family, for the first time in his life.
The Potters smiled and waved at Harry and he stared hungrily back at them, his hands pressed flat against the glass as though he was hoping to fall right through it and reach them. He had a powerful kind of ache inside him, half joy, half terrible sadness. (12.127-128)
This passage really emphasizes how much Harry misses the parents he lost and how badly he wants to know something, anything, of them. This is the first time he's ever seen their faces; no wonder he wants to "fall right through [the glass] and reach them." Even though "half" of his feelings are "terrible sadness," it's worth enduring that for the "joy" he feels in looking at them at long last.
Your mother died to save you. If there is one thing Voldemort cannot understand, it is love. He didn't realize that love as powerful as your mother's for you leaves its own mark. Not a scar, no visible sign… to have been loved so deeply, even though the person who loved us is gone, will give us some protection forever. (17.120)
Even though Harry's parents are gone, in a way they're still with him, "mark[ed]" on his very skin. In a way, there's no greater measure of love than giving your life for another person, and that's what Harry's mother did for him. That kind of sacrifice forms a magic spell more powerful than any other in this novel.