Entropy is physics-speak for mess: it's a measure of how much disorder is in a system. The Second Law of Thermodynamics says that the entropy of the universe is always increasing. Let's let Valentine explain:
VALENTINE: Your tea gets cold by itself, it doesn't get hot by itself. [...] Heat goes to cold. It's a one-way street. Your tea will end up at room temperature. What's happening to your tea is happening to everything everywhere. The sun and the stars. It'll take a while but we're all going to end up at room temperature. (2.7)
Scientists didn't figure this out till 1824, so for Thomasina to get it in 1812 means that she's well ahead of the game. But the Second Law isn't in Arcadia just to show off how smart Thomasina is: it's connected to time, which, like heat and jam in rice pudding, can't go backwards. The Second Law describes scientific observations, but it also resonates with emotions like regret and nostalgia, the impossible desire to go back and reorder the past, not to mention fear of a chaotic future. Arcadia's emphasis on the Second Law of Thermodynamics brings together Classical science and Romantic feeling in intriguing ways – not what you'd expect from a page in your physics textbook.