How we cite our quotes: (Line)
Quote #1
Ornaments would mar our union;
they would come between thee and me; (3-4)
Literary "ornaments" don't bring the speaker closer to God, they distance him from God. These lines tell us that the speaker is seeking some kind of divine union. He wants to get up-close and personal with the Big Man Upstairs (God).
Quote #2
their jingling would drown thy whispers (5)
Fancy-schmancy poetic language "jingl[es]." That is, it distracts us from God's message. And the speaker really wants to hear God's message. The idea here is that, as poets, we should be focused on meaningfully connecting to the divine by trying to listen to his message. Only if our words are simple are we able to "hear" the divine whispering to us.
Quote #3
My poet's vanity dies in shame before thy sight.
O master poet, I have sat down at thy feet. (6-7)
These lines suggest how much of an inferior poet the speaker is, compared to God. The real "master poet" is God, and the speaker's powers of poetry pale in comparison. It seems like an unfair comparison, though, don't you think?