Normally, we'd say a book is tough because of its vocabulary, or because of its writing style, or even because of the historical information needed to understand it. But that's not what's going on here: Ursula Le Guin is pretty over the top in explaining the philosophy behind her work, and the writing is clear and simple.
Nope, what might trip you up is the fact that reality is constantly shifting right underneath your feet. It's hard to keep up with what's changing, and who's there, and if the things you remember are from one reality or another. It's so confusing that even the characters have a hard time keeping it straight. We're sure that feeling of uncertainty is completely intentional, but it still makes The Lathe of Heaven a little more difficult to read.