In our "Writing Style" section, you'll notice that we kind of go head over heels for Joyce's style. And not just because it's beautiful. We have included Language here again in "Symbols, Imagery, Allegory" because it's important that you think of the language in the book the same way that you think of, say, the book's correlations to the Odyssey.
In "Proteus," Stephen is walking along Sandymount Strand, and as he looks down the beach, he thinks, "These heavy sands are language tide and wind have silted here" (3.62). It's a very important quote to Ulysses. Namely, language has a physical presence in the book: it's the material of Joyce's world. And you thought that the world was made of atoms? Well, in a piece of literature, words are your atoms.
The philosopher Ludwig Wittgenstein was notoriously difficult to understand, but one of the things he said was: "The limits of your language are the limits of your world." Joyce was a student of languages (he was fluent in four or five), and was absolutely obsessed with words. It was like the larger his vocabulary got, the more material he had to work with. But the idea gets a little more complicated because the way Joyce thought, language mediates our relationship with the world. It determines what we are capable of thinking and experiencing. For example, if we didn't know the word for "love," we wouldn't be able to think about the idea of love. Love would just be a physical thing, a certain intensity in the chest, but we wouldn't be able to think of it as anything beyond that, to speculate on its nature and so on and so forth.
If this idea is correct that language defines the limits of your world, then Joyce's world was more unbounded than perhaps any writer before or since.
But why is it that it's the "heavy sands" that are language. Well, Joyce's other idea was that our language had become rigid and calcified (think of it turning hard like bone). People had said the same things over and over again so many times that they had ceased to express anything, to carry any feeling. In a language full of clichés and stock turns-of-phase, Joyce felt that words themselves had become flat and powerless.
A great way to think of this is that language is one great big organ (the instrument). Joyce sits down to play it, and though not too many other people notice, he can't help but feel that it's horribly out of tune. So Joyce uses all of his talent to try to tune and re-tune the organ until it plays to his satisfaction and can create beautiful music once more.