You can't really talk about the Puritans without talking about religion. The Puritans came to Massachusetts in 1630 to found a place where they could practice their particularly strict form of Protestantism. The strictness of Puritan religious faith led them to persecute other people who didn't share their religious views, and it's this intolerance and self-righteousness that Hawthorne targets in The House of the Seven Gables. Interestingly, though, Hawthorne doesn't set the novel in the 1690s; it takes place in the 1850s. So he's not just interested in original Puritan oppression, but also in its legacy in the present day.
Something else we find intriguing: the Puritans believed in the Calvinist idea that God has already chosen who is saved and who is damned. In this religious system, it's all about fate, not free will. Thus the idea of fate isn't just important to Hawthorne personally; it's also a big deal for his social context more generally. What's more, this doctrine of predestination promised that even though we can't know for sure during our lifetime whether we are saved, we can guess by looking at our material success on earth. The richer you are, the more likely it is that you're saved, or one of the "elect." Clearly such a religious belief would appeal to Judge Pyncheon.
Phoebe's regular churchgoing demonstrates her participation in the larger social context of the town, while Hepzibah and Clifford's lack of interest in religious observance underlines their general isolation from the rest of town life.
Hepzibah's private pleas to God for help provide a model for genuine religious faith, while Judge Pyncheon's extensive records of church attendance casts doubt on the sincerity of his religious belief.