When Sam made Hally the kite, he helped him fly it but then tied it to a bench and left him alone to watch his kite. At first it isn't really clear why he'd do such a thing—why spend all the time making a kite and then miss out on the fun of flying it? Hally didn't understand:
HALLY. You explained how to get it down, we tied it to the bench so that I could sit and watch it, and you went away. I wanted you to stay, you know. I was a little scared of having to look after it by myself.
SAM. (Quietly) I had work to do, Hally. (885-890)
If you thought there was something meaningful in Sam's quiet answer, you were right. We learn the truth later on in the play:
SAM. I couldn't sit down there and stay with you. It was a 'Whites Only' bench. You were too young, too excited to notice then. But not anymore. If you're not careful…Master Harold…you're going to be sitting up there by yourself for a long time to come, and there won't be a kite in the sky. (1840-1850)
Sam didn't want to trouble Hally with this info when he first asks about it. Only after Hally's ugly outburst does he explain it all. He sees Hally falling into a racist mindset and wants to warn him that it will just lead to more shame and no kite to rescue him from it.
SAM. You know what that bench means now, and you can leave it any time you choose. All you've got to do is stand up and walk away from it. (1783-1787)
The bench is a perfect symbol of apartheid and the kind of damage it does to people. It isolates them.