If we had to describe the speaker of the "The Altar" in just one word, and one word only, that word would be… religious. If we could have two words, we would say super-religious. If we could have three, we would say, well… super-duper-religious. Check it out: he writes a poem called "The Altar" that also happens to be in the shape of an altar. On top of that, there are biblical references all over the place (Exodus 20:25, Psalm 51:17, Luke 19:40, to name just three).
It's not just that the speaker knows his Bible really well, or can make cute little shapes that have religious significance, although this is part of it. He's also a very devout man, one dedicated to making sacrifices for, and praising, his god. Things do get a little confusing, that's for sure, but clearly the speaker has built an altar that is both the poem itself (it's in the shape of an altar, remember), and the speaker's heart: "each part / Of my hard heart meets in this frame / To praise thy name" (9-11). And the poem-altar will continue to praise God long after the speaker ceases to exist: "That if I chance to hold my peace / These stones to praise thee may not cease" (13-14).
Another sign of the speaker's devout religious beliefs is the fact that he really goes out of his way to make sure his heart-poem-altar complies with all of the different instructions in the Bible. In Exodus, for example, God specifically says that altars should not be built out of "hewn stone," while in the book of Psalms the singer says that the proper sacrifices for God are a "broken and contrite heart." The speaker offers both a "broken" heart and an altar of unhewn stone.
Knowledgeable about the Bible? Check. Super-duper-devout? Check. What else can we say about the speaker? Well he's got a knack for creativity, that's for sure, and he's well aware of his imperfections. All that stuff about broken hearts and tears is his way of recognizing that he's a sinner. He knows God will not judge him for this, and that all he has to do is offer up his heart just the way God made it (imperfect).