Symbolism, Imagery, Allegory
There's an awful lot of debt, both literal and figurative, in Middlemarch: Fred Vincy gets himself into trouble by persuading Caleb Garth to co-sign a loan that he isn't able to repay, Lydgate falls into serious debt after his marriage to Rosamond, Will Ladislaw hates feeling indebted to Mr. Casaubon, and Mr. Farebrother enjoys feeling indebted to Lydgate for recommending him for the new post at Lowick.
Fred's debt to Caleb Garth is both literal (he owes the man money) and figurative (he's later obliged to him for his trust in giving him Stone Court to manage and his daughter to marry). Fred's debt actually helps him mature as a character – it's a yoke, but it's one that he needed to make him settle down and work for a living. Owing money to a stranger (Mr. Bambridge) didn't provide him with any incentive to work hard and pay it off, but owing money to Caleb, the father of the girl he loves, certainly does.
Lydgate's debt is also a yoke, but it's not a productive one. It oppresses and hampers him. And when that debt, like Fred's, is transferred from the hands of anonymous bankers to Mr. Bulstrode, someone he knows, the yoke becomes still worse. Being indebted to Mr. Bulstrode turns out to be the worst thing that could have happened to Lydgate, so Dorothea generously offers to take on the debt herself.
Debts in Middlemarch are always getting transferred from person to person, so that characters are always indebted (either literally or figuratively) to someone new. So one way of looking at debt in Middlemarch, both literal and figurative, is that it ties characters together – indebtedness reinforces the intricate "threads of connection" that form the social (and economic) "web" of the novel.