Did we just say that dissatisfaction is the glue that binds Sonnet 75 together? Shouldn't we have said suffering, instead? Well, we could have—but the two ideas actually go pretty much hand in hand. The easiest way to tell how much the speaker is suffering is from the negative Nelly attitude he projects in the poem. Just look at his malicious delight in turning pretty things into their opposites. Based on the poem as a whole, we can guess that this hostility is his way of lashing out and relieving the suffering that his unsatisfying relationship has made for him.
The speaker's suffering is directly connected to his dissatisfaction, so really, he's to blame. Take that, Mr. Speaker.
The poem does not give us any indication that "you" is suffering, too. Suffering here is one-way street for the speaker, with nothing but green lights.