"…she's the first person who's ever belonged utterly to me."
"She belongs to me, too."
"No, you have two other children. She's mine."
"Great balls of fire!" said Scarlett! "I had the baby, didn't I? Besides, honey, I belong to you."
"Do you, my dear?" (50.101-105)
Rhett seems to see love as meaning that someone else belongs to you, so he's upset with Scarlett because she doesn't completely belong to him. How is that different from Frank being all cranky because Scarlett has interests outside the home, like the mills? You sort of wonder whether Rhett deserves Scarlett's love any more than Ashley does.