Here's why Thomas doesn't want his darling baby girl marrying David:
THOMAS: Remember the chauffeur on the Harrington estate? His daughter fell in love with the son. Next he was driving the family to the church, changing his uniform and then giving the bride away. That's not for me. I don't like it.
On the one hand, you could see this as simple kneejerk classism. Thomas doesn't like the classes mixing; each should stay in their own place.
But you could also see the story as a description of how rich people suck. Think about it; they had the father of the bride drive them to their wedding—he had to work on his daughter's wedding day. That's humiliating, and just plain mean. Why would they do that?
Well, one reason they might do it is because they don't think of the lower classes as fully human. And, in fact, Linus treats Thomas with similar contempt; he has the chauffeur drive him and Sabrina out to their dates, and doesn't realize how weird and disrespectful that is until Thomas points it out.
Linus calls Thomas a "snob" because he doesn't want his daughter dating someone so much wealthier than her:
THOMAS: I like to think of life as a limousine. Though we are all riding together, we must remember our places. There's a front seat and a back seat and a window in between.
LINUS: Fairchild, I never realized it before, but you're a terrible snob.
THOMAS: Yes, sir.
But he's not a just snob. He also doesn't want his baby girl getting hurt.
Thomas has got serious, reasonable concerns about the way the Larrabees behave. As it turns out (as far as we see) his worst fears aren't realized. But they aren't foolish fears. Thomas cares about his daughter, and he knows Linus and David well enough not to trust them. The movie doesn't quite want to admit it, but Daddy is right.