He feels like a "sneak" by not admitting to Lydgate that he'd been there that morning, but he can't see a way around it.
Lydgate tells him about all the problems they've had since he left Middlemarch (of course, Will has heard the whole sob story from Rosamond that morning).
Lydgate concludes the story by saying that they're going to have to move to London – which is what Rosamond wants, and what he absolutely doesn't want.
Will feels terrible. He hasn't done anything wrong, but it looks bad that he should have spent so much private time with his friend's wife. Especially since she obviously thinks he was in love with her.