This poem sets up a kitchen scene. We see an icebox, plums, and we can imagine the later breakfast (sans stone fruit). The poem itself is probably a note on the icebox. This image shows us a lot about the life of our speaker, and about his relationship to the other person who lives with him, although we can't always pin it down. He is close enough to her to eat her plums, but loves her enough, and feels bad enough, that he leaves this note. What she has to say about her missing fruit, we can only guess.
Lines 2-4: These two people are firmly established in this home, because they keep plums in the icebox for later.
Lines 6-8: The speaker knows his housemate—who could be his wife, or his mother, or just a roommate—pretty well, if he knows that they are the type to save plums for breakfast. But, he has eaten the plums anyway, the jerk! These lines also let us know that the people who live in this house have different, incompatible schedules. They don't eat at the same time, and have to communicate with notes like this, or at least they do today.
Line 9: It's a good step for our speaker to ask for forgiveness, and it's something that, in many homes, doesn't happen enough. This shows that he feels deep guilt, and probably cares about the person he's leaving the note for. He feels bad about robbing her of the experience of eating these delicious plums.