Get ready to get dirt under your fingernails. The speaker is the son of a farmer. He admires his father, and spends much of his young years following him around the field, stumbling and chatting, trying to keep up while his father works diligently. You can see the admiration for his father, especially in the earlier parts of the poem: "The horse strained at his clicking tongue./ An expert" (4-5). He goes on to further explain how crazy skilled he is that he can turn over sod without breaking it, and he can manage his team like a pro: "The sod rolled over without breaking. At the headrig, with a single pluck/ Of reins the sweating team turned round" (8-9). He's the baddest in the biz, and his son is full of admiration.
Most of the poem is told in the past tense, as the speaker looks back to his childhood, and his relationship to his father and his father's work, but at the very end, we're snapped into the present tense. Now, the young boy has grown into a man, and the tables have turned: his father follows him around all the time, and he can't seem to find a way to shake him. The final lines read: "But today/ It is my father who keeps stumbling/ Behind me, and will not go away" (23-24). The speaker seems to have shaken the desire to be like his dad and is a little taken aback, a touch annoyed that his dad follows him around now.
In the end, then, the speaker's relationship with this dad is the real focus of this poem. Their reversals are complete, an now it's Dad's turn to stumble and be annoying. This seems to speak to a deeper rift between father and son. There must have been a time when they were on equal footing (neither one stumbling or annoying), but they seem to have missed it in this poem.
But don't bust out the Kleenex just yet. It seems that last phrase—"will not go away"—also offers the possibility of comfort, right? However they annoy one another, our speaker and his father still have each other. Silver lining alert, gang.