Everyday, Dimmesdale has to wake up and receive the adulation of his community for being basically a saint on earth. It's exhausting.
But seriously, Dimmesdale is living a lie—and that is exhausting. You have to be on your guard constantly, so no one finds out that you're lying about only liking Nickelback ironically. In The Scarlet Letter, hypocrisy is one of the worst sins that a man can commit. Just as adultery produces a physical mark on Hester's body (the baby), hypocrisy produces a physical mark on Dimmesdale's body. And only Pearl can see through him—so, when he finally confesses, she can love him for who he is: her father. In the end, our reputations are less important than our lives. (Maybe. Except when it comes to Nickelback.)
Hypocrisy is sometimes necessary. Dimmesdale destroys himself through hypocrisy, but his life is a blessing to many in the community.
Even though hypocrisy appears to save Dimmesdale from punishment and humiliation, his torment is worse even than Hester's.