The church bells only appear once in the poem, but they manage to rock Coleridge's world with their prophecy about some glorious future, be it a life in heaven or a spiritual life on earth. They're poignantly described as "the poor man's only music" (since the poor typically didn't have access to pianos in their homes or symphony orchestras, and MP3s obviously didn't exist in Coleridge's time) and as "articulate sounds of things to come." Even though the sound of church bells isn't words, it somehow is as "articulate" as human language in describing the glory of God's Kingdom (or is possibly more articulate).
Lines 27-34: Occurring during the discussion of the "stranger" in the fireplace, the church bells also make Coleridge imagine some coming glory. That glory turns out to be the revelations hidden in Nature—which Coleridge wants his son to be able to access more easily.