If you really wanted to, you could say that "Church Going" is about the tension between religion and spirituality. Go ahead! Just say it. Religion here refers to the "official" answers that spiritual faith gives to those big life questions like, "Why are we here?" In contrast, "spirituality" tends more to ask questions. In this poem, you have a speaker who only has a slight knowledge of religion, yet this knowledge has a huge impact on him because he often wonders whether or not he's approaching spiritual questions in the "right way." Ultimately, it doesn't look like he can get behind religion, but he (and the poem as a whole) definitely admits to the appeal of religion, especially for people who want to find something in life that's worth taking seriously.
Philip Larkin is straight up hating on religion here. "Church Going" is completely ironic in tone, even when it's supposedly admitting to the so-called appeal of religion.
Oh, touché. Through his use of poetic form, Larkin suggests that whether we believe in religion or not, there is a higher power governing our lives.