Where It All Goes Down
We might imagine that the speaker is standing over his son's grave, but if you look closely, there's no evidence in the poem that tells us that. Really, the only thing we can say for sure about the setting is that it's a kind of conversation.
"Um, how can a conversation be a setting," you ask? Well, we got answers: since there really is no direct description of a physical place, we can instead think about the form in which this poem takes place. In this case, we're not talking about the iambic pentameter or that kind of stuff. Instead, we mean the general structure of how the poem is put together.
For instance, notice how in the first four lines the speaker directly addresses the son ("Farewell, thou child…"). Noticing now? Good. Then, in the next four lines after that, he turns from talking to his son to talking to…well, who do you think? It seems here that he's kind of shouting at the sky, talking to whomever might listen to him ("could I lose all father now!"). At the very least, then, he's talking to us, his readers. Then, in the final four lines, he turns back to address his son again ("Rest in soft peace").
Now, what's interesting about this conversation is that it's entirely one-sided. Usually, when you have a conversation, somebody is there to talk back to you, right? But in this case, it's just the speaker by himself. His son is dead, and nobody else is in the poem to listen to him. So…just who in the Sam Hill is he really talking to?
One answer is "all of the above." He's talking to his son still because he loves him and wants to keep him alive in his (the speaker's) memory. He's also talking to us, his audience, because he wants to share his thoughts on death with us. Most importantly (we would say), this guy is talking to himself. More than anything, this poem represents him trying to make sense of what is really an utterly senseless event: the death of his child.
So, while we get to listen in, his true audience is himself. He's talking this out in order to move on. And, by gum, it works. By the end of the poem, he seems to have a strategy for moving on.