The speaker in this poem is a writer, quite possibly a poet. He's the son of a potato farmer, and as we quickly find out, he's the grandson of a harvester as well. He comes from a long line of diggers. But what else do we know about him, really? We're left only with the guesswork of his thoughts and feelings. We'll have to use his words to figure him out.
Mostly, he talks with the utmost respect about the hard-working attitude and physical strength of his father and grandfather, which they demonstrated day in and day out in their work. By the deferential (super-respectful) way the speaker talks about them, you'd think they were his heroes. He is both full of praise for them, and a little hard on himself (he doesn't quite think his work as a writer is as important as his father's/grandfather's). So that tells us he respects hard work, is proud of his family, and wants to be like them so much that he strives to find a connection between his writing and their manual labor.
It would be easy, and not much of a stretch, to assume that the speaker is Heaney himself. Heaney, after all, is a writer, and the son of a farmer. But because he never names himself, we don't have the proof. We'll just have to satisfy ourselves with the mystery. Hmph!