Borrowers don't like humans, humans don't like borrowers, and "borrowing" in one world is "stealing" in the other. Ugh. Can't we all just get along? The inability of big and little people to get past differences ruffles a lot of feathers in The Borrowers, and those who struggle to overcome them mature into more developed characters. So hey, at least it's worth the trouble.
Mrs. Driver's actions (taking handkerchiefs, wine, and other such things) are no different from what the Borrowers do, so she should probably get a life and stop bugging 'em.
Mrs. Driver's actions are worse than what the Borrowers do; she is actually stealing.