You might think that such a fan of communism, which has a historical rap of suppressing religious expression in any form, would be less religious in his tone. That's not the case for this poem's speaker, though. While he's not bigging-up the practice of religion in any orthodox sense, he does borrow on the idea of religious prayer as a way to describe the work of his poem. Beyond that, our speaker is definitely a spiritual fella, who is looking to achieve for his country the same kind of purity of spirit that is the goal of any spiritual, or religious, life.
Line 8: The speaker wants to know when America will be "angelic." Now there's an adjective. While it refers literally to those holy beings with the chubby cheeks and the feathered wings, we take this to be more of a metaphor for how the country should behave. Sure, angels are cute (at least the ones we think of are), but they also reflect a purity of spirit and a goodness of intention. This is what the speaker seems to be striving for in his country.
Line 16: When the speaker says, "America after all it is you and I who are perfect not the next world," we take this is a statement of irony. After all, how can a land filled with murder and materialism, and a man who freely admits that his mind ain't right, be more perfect than heaven, the "next world"? Here he uses the idea of a next world, which is typically thought of as a perfect place where we go if we've been good in this world, to contrast the terrible state of the world that the speaker is in—himself included. He also criticizes the complacent attitude that would just accept the world as it is, and not try to do anything to perfect it.
Line 18: Hey, America! "You made me want to be a saint." Here the speaker confesses that he, at one point in time, was inspired by the example of America. And not just a little inspired either. He was once so taken with America's goodness that he aspired to be a saint, one of the purest of the pure spirits. Once again, the speaker uses the idea of spiritual or religious perfection as a both a personal, and a political goal. Sadly, he also shows just how far reality is from that goal.
Lines 35-36: Okay, so the speaker's not totally into every aspect of religion. For one thing, he "won't say the Lord's Prayer." We don't see that as rejection of religion or spirituality altogether, though. Instead, it's another way that the speaker shows us that he's not buying into the system of religion. He refuses to behave in an orthodox, expected fashion, but that refusal in some ways allows him the freedom of perspective to look back at what passes for normal, and critique it.
Instead of following society's lead, then, the speaker has "mystical visions and cosmic vibrations." He wants us to know that, even though he doesn't toe the line of organized religion, his spiritual fervor is no less intense.
Line 53: Did you know that John F. Kennedy was the first Catholic president? When he was running for his eventual election in 1960, Kennedy's religion was a big deal to some, who thought he might be more loyal to the Pope in Rome than to his own country. Just a few years before Kennedy's election, though, this speaker lets us know that he identifies with the future president: "My ambition is to be President despite the fact that I'm a Catholic." The speaker seems to identify with Kennedy's status as an outsider, due to his religious practices.
Line 54: The speaker's got a question: "America how can I write a holy litany in your silly mood?" A litany is a kind of prayer in which a list of things are mentioned or asked for. In this way, we might also see the poem itself as a kind of litany. Think about it. Whether it's an abstract concept like God or America, doesn't addressing it directly count as a kind of prayer? It's kind of a moot point, though, since for the speaker America is just too much of a goofball to pray to.