Who'da thunk Joseph would be so interested in a piece of gold? What's gold to this goody-two-shoes, right?
Yeah, well, this particular piece of gold is a special symbol that reminds Joseph of Fanny. When he's robbed on the road, he begs people to "search for a little piece of broken gold, which had a ribband tied to it, and which he could swear to amongst all the hoards of the richest men in the universe" (1.14.10).
Later on, the folks at the inn try to confiscate the piece of gold to use as evidence against the robbers who left Joseph in a ditch. Great, right? Not right: we find Joseph practically in tears as he tries to prevent his piece of gold from disappearing in a lengthy trial. More than revenge or justice, Joseph values his relationship with Fanny. If he let this piece of gold vanish into the ether, his value system would be totally out of whack.
In a nutshell, Joseph is keen on recovering the piece of gold because it represents Fanny and his love for her. Joseph doesn't care how much the gold is worth in monetary terms; it's only meaningful to him as a reminder of Fanny. On top of that, Joseph is struggling to keep his virtue intact. He tells us he's resisting temptation so that he can save himself Fanny, so it makes sense that he needs a little reminder now and again.