This is a poem about the awesomeness of simple poetry, so is it any wonder that the language and the diction of the poem are pretty simple? "Song VII" doesn't sound like "Poetry" with a capital "P" as much as it resembles a guy just talking out loud.
True, there are some formal elements to the language (like when the speaker refers to God as "thee" in line 4). But generally the poem sounds pretty conversational. If we take the first two lines for example, "My song has put off her adornments/ She has no pride of dress and decoration," we can see that the language is quite simple. It flows easily, and it sounds more like prose than poetry, as there are no strong poetic rhythms or techniques to be found. Given that this poem is all about how that kind of stuff keeps you separated from God, this all makes good sense to us.