Aww, the poor Clerk. Literally. This guy's so poor that he can't even afford to feed himself, let alone his horse, who's as skinny as a rake. The Clerk's clothes are threadbare on his emaciated body. But it's not necessarily for lack of money – it's just that when the Clerk has money (usually after borrowing from friends), he prefers to spend it on books rather than food or clothes. He's so serious about his studies that he even appears to be studying on pilgrimage – an antisocial behavior for which the Host calls him out later on. For this (and for choosing books over food) he appears a little ridiculous.
The narrator tempers his satire of the Clerk by also telling us that he diligently prays for the souls of those who lend him money for books and lessons, that he speaks little but what he does say is always virtuous, and that "gladly wolde he lerne, and gladly teche" (General Prologue 310). Kind of like us here at Shmoop, this guy's generous with his knowledge. Since this attribute was one of the most desirable for medieval scholars, we have permission to think well of the Clerk. And don't feel too sorry for this guy – his fortunes could take a turn for the better if he gets a benefice – a position as a priest that comes with a salary.