Though we don't see much about the speaker's family or home, "This Is Just To Say" takes place in a kitchen, and we can presume it takes place between two people who live together. In fact, we might think of this poem as a simple reflection on the way that, in a home, people's lives can overlap. One person eats something of the other's and apologizes. These little everyday things end up contributing to the bonds of a home in a big way.
The domestic life in this poem is an unhappy one. Why can't the speaker just apologize to the other person's face?
The poem shows that the speaker is plumb in love with the person with whom he lives. Pardon the pun.