The first thing we learn about Ella's father is that he was away on a trading expedition when she was born (1.3). This sets the tone for the rest of the novel, because we learn (1) that he's basically always away on a trading expedition, and (2) that he's extraordinarily good at what he does.
In fact, he's maybe a little too good. The sort of good that might mean you're fudging numbers a wee bit.
This comes back to bite him on the butt when he sells an estate that doesn't actually belong to him—oops—and then has to repay the wronged parties. To settle his debts, he has to sell the family's belongings, including Ella. (In a sense.) We get the impression that he doesn't particularly care if he sells his daughter into an unhappy marriage so long as it's a rich one.
In addition to being the tiniest bit unethical, Sir Peter is in business for love of money, and it shows. After his failed attempts to obtain valuable sculptures from the elves, Slannen tells Ella: "Sir Peter is a witty man and a shrewd trader, but if he had admired our things as you do, we would have been gladder to let him have them" (13.63). Seems to us that the only thing Sir Peter admires is money. (He'd probably get along great with capitalist fat cat.)
Sir Peter doesn't take an active hand in Ella's upbringing until she comes of age to marry, and then he sends her to finishing school. Not exactly what you would call hands-on parenting. And, okay, he does eventually blame himself—but not for being absent. For letting his daughter grow up "an oaf" (5.17).
After marrying Dame Olga and being hit with an eternal love spell, Sir Peter still has the self-preservation instinct to get out. Even when that means abandoning Ella to cruel treatment at her stepfamily's hands. Nice job, Dad.
We get the impression that Sir Peter is not a nice man. Well, it's not an impression so much as a simple fact. Even the servants gossip about it, and Sir Peter knows it. He even seems to gloat about it, telling Ella: "Your father is not a good man, as the servants have already warned you … They may have said I'm selfish, and I am. They may have said I always have my way. And I do" (5.32-34).
Okay, okay, we get it: don't mess with him.
He's not above subtly threatening his own daughter, too. We know Ella's no coward, but even she acquiesces to go to finishing school rather than risk his wrath: "The anger in his eyes was so tightly coiled that I didn't know what would happen if his spring were tripped" (5.37). Sounds like the dude might need some anger management.
And, finally—what, you didn't need another reason to hate him?—his self-interest leads him to marry Dame Olga even though he doesn't seem to actually have feelings for her. When Lucinda "blesses" them with eternal love, he's "open-mouthed with horror" (21.1). He decides to love Dame Olga from a distance after that, so that he's alternately tortured by his longing for her and relieved because he's not that fond of her after all.
Basically, Sir Peter is interested in his own welfare above all else. Okay, fine. We just wish he were a little more interested in his own daughter—and not just as another commodity.