A fly on the wall—that's the best way to describe the speaker of this poem. Okay, he's not literally a fly, but he's like a fly on the wall because he's watching this whipping unfold and, it seems, the woman and the boy have no idea that he's watching.
If we had to guess, we would say that the speaker lives near the woman and the boy, perhaps in an apartment across the way, from which he can see everything that happens. He obviously is familiar with the vegetation in the other place (he describes elephant ears and zinnias) and knows which room belongs to the boy.
More than just a nearby, casual observer, fly-on-the-wall-type dude, the speaker of "The Whipping" has a knack for psychology, and definitely understands how the cycle of violence and abuse works. In the last stanza, for example, the speaker suggests that the woman whips the boy because of the pain, suffering, and abuse she endured in her past—she is "avenged in part for lifelong hidings." She beats the boy as a way to cope with her own pain, to "purge" or eliminate it from her life. In fact, this guy is so good, he can practically read the woman's mind. Just look at that bizarre little stream of consciousness interlude in stanzas four and five, where a series of painful, violent memories are vividly described. Yep, it sure sounds like he could give our boy Sigmund Freud a run for his money.
Now, there's one last little thing we need to cover. We've been talking about how the speaker understands violence and might as well be a therapist. Those fourth and fifth stanzas are a little confusing, and it's possible that the memories vividly described there are the speaker's own recollections. He too could be a victim of abuse, and perhaps the scene he watches unfold reminds him of his own violent childhood. Perhaps this is why he understands the cycle of violence and abuse so well (just sayin').