The plot of Silas Marner is simple and the writing some of Eliot's clearest, but the dialect can be tricky. Eliot was careful to represent country language accurately, as in this gobsmacker of a passage:
For there's my wife, now, can't smell, not if she'd the strongest o' cheese under her nose. I never see'd a ghost myself; but then I says to myself, "Very like I haven't got the smell for 'em." I mean, putting a ghost for a smell, or else contrairiways. And so, I'm for holding with both sides; for, as I say, the truth lies between 'em. And if Dowlas was to go and stand, and say he'd never seen a wink o' Cliff's Holiday all the night through, I'd back him; and if anybody said as Cliff's Holiday was certain sure for all that, I'd back him too. For the smell's what I go by. (1.7.58)
Here, the landlord of the Rainbow Inn is saying that the fact that he's never seen a ghost doesn't mean that they don't exist, because he simply might not have an eye for ghosts, the way his wife doesn't have a good sense of smell. But we forgive you for not understanding that the first time. We had to read it a couple times ourselves.
If you can push through the dialect (there's a lot in chapter 6) and some of the heavy philosophical narration (although much less than in Eliot's other books), the rest of the story is pretty smooth sailing.