Hard by yon wood, now smiling as in scorn, Mutt'ring his wayward fancies he would rove, Now drooping, woeful wan, like one forlorn, Or craz'd with care, or cross'd in hopeless love.
This stanza continues with what the speaker imagines an old villager would say about him after he was dead and gone.
He imagines the old guy saying that the speaker used to rove, or wander, in the nearby woods.
Sometimes, the speaker would smile almost scornfully, while muttering to himself, and sometimes he would look all droopy and mopey, pale ("wan") with sorrow, like he was anxious or else hopelessly in love with someone who didn't love him back. Good times!