There's not a whole lot of scenery specified in The Zoo Story. But the one specific thing Albee calls for, the one thing you absolutely have to have if you're going to stage The Zoo Story, is "two park benches, one on either side of the stage," as the stage directions tell you.
The bench is all the scenery—and symbolically in the play, the bench is all the everything, too. Peter has a good job, a loving family, a nice home, and some pets. The bench symbolizes his comfort, his stability, all the things he has and enjoys happily:
I sit on this bench almost every Sunday afternoon, in good weather. It's secluded here; there's never anyone sitting here, so I have it all to myself. (217)
Peter's got a routine, he's got good weather, he's got privacy. He's happy and on the bench. So when Jerry sneers at Peter and tells him:
You have everything, and now you want this bench. Are these the things men fight for? (247)
Well, it's a little deceptive. Yes, Peter has everything, and doesn't need the bench. At the same time, the bench stands in for everything. If they can take your bench, what will they take next? Your parakeets, your spectacles? Your home, your spouse, your job?
Peter is willing to fight for the bench because the bench represents all that stuff. And when at the end he loses the bench, when he's "dispossessed" (278) by Jerry's death, that means his whole life has been upended too. Nothing is as safe as it should be. When Peter runs offstage at the end, he rushes into a less happy, less secure world, where you can't count on money or family—or even benches—to protect you.