Thanks to the use of dialect, not to mention the subject matter, we know this story (like many of O'Connor's) is set in the South. The tension in this setting is perfectly summed up in the first page when we learn that Julian's mother "would not ride the buses by herself at night since they had been integrated […]" (1).
Talk about tense. After a decade of desegregation, the 1960s had mostly succeeded in integration. On the surface. Blacks and whites could ride the same buses, attend the same school, and hold the same jobs. But as this story shows us, things aren't always what they seem.
On a more micro level, most of the action of the story is set on a bus. We have some ideas for why O'Connor chose this setting, and we can bet you could come up with a lot more:
Movement (check out the bus in "Symbols" for more on this). Yeah, in a literal sense because, well, buses have wheels, but also movement in time. The passengers are moving forward with integration and civil rights), even if the ride is a little bumpy.
Entrapment. There's a high level of claustrophobia in this story, like this: "Meanwhile, [Carver's mother] was bearing down upon the empty seat next to Julian. To his annoyance, she squeezed herself into it" (79). We've all been in this situation—you're on a bus (or in a car, or a plane), and you can't get out.
And one more thing. Being taken on a ride that you can't escape? That sounds a lot like the kind of journey that Julian's mom is on, to us.