The House of the Seven Gables is a family drama, and one of the primary ways of defining a character is through his family life. Hepzibah devotes herself completely to her brother's health and happiness, so we know that she is a good person (even if she doesn't look like one). Judge Pyncheon, by contrast, is willing to replace his deceased wife's cracked tombstone only because he's glad she's dead. He thinks she was foolish and "oozy" (18.7) with tears. This is not a sensitive man.
Appearance is tricky in The House of the Seven Gables. As we observe in "Characters," Judge Pyncheon looks kind but isn't, and Hepzibah looks mean but isn't. But sometimes appearances can be strikingly accurate. The stern portrait of Colonel Pyncheon shows us exactly what kind of a man he is. And Judge Pyncheon's close resemblance to Colonel Pyncheon in his daguerreotype tells us everything we need to know about his character. So physical appearance is important, but only when it's attached to a symbolically important object: a painting, a mirror, or a photograph, for example. Fortunately Hawthorne gives us plenty of symbols to make this mode of characterization work.
Any book that's as concerned about hypocrisy as The House of the Seven Gables is going to insist that you check out a person's actions rather than their words. We hear Judge Pyncheon offer Hepzibah and Clifford a place to stay – so kind! – but we also see the flash of his furious expression when Hepzibah says no. We hear Judge Pyncheon express his concern for Clifford's welfare, but then he refuses to leave him alone, even when the poor man begs for mercy. Clearly Judge Pyncheon's actions should be speaking a heck of a lot louder than his words.