How we cite our quotes: (Line)
Quote #1
For God's sake hold your tongue, and let me love; (1)
God is the second word in this poem. In this first line, in fact, we have the speaker's argument in a nutshell: his love is actually for God's sake. It brings him up to heaven.
Quote #2
And by these hymns all shall approve
Us canonized for love; (35-36)
The switch from poems to "hymns" is quick, subtle, and significant. One minute the couple are being celebrated in non-religious writing (like sonnets), and the next they are being held up in religious songs. This pivot allows Donne's speaker to project himself and his lover into sainthood in the poem's concluding stanza.
Quote #3
And thus invoke us, "You, whom reverend love
Made one another's hermitage; (37-38)
A hermitage is a religious retreat, one that religious figures (a.k.a. hermits) would go to in order to be alone and contemplate God. Here, though, love is the foundation for that kind of spiritual experience. The lovers get the same experience by being in love. This seems like almost a blasphemous idea, but if God is love, then maybe this is just the logical extension of that concept.