The first thing to notice in "Famous Blue Raincoat" is that in the initial verse and in several other places, the lines are actually written in poetic meter. That's right; Cohen's counting syllables. He uses what is called an amphibrach, where each poetic foot (group of syllables) is composed of a stressed syllable sandwiched between two unstressed syllables. When you listen to him sing those first lines, you'll find that his voice lilts up and down in perfect agreement with the stresses: "It's four in | the morning, | the end of | December."
For much of the song, the lines are twinned in their rhymes in the basic structure of AA, BB, CC, etc. As often as not, the rhymes are slant rhymes, meaning that they're not complete rhymes, but that there's still consonance (similar consonant sounds) on the final consonants of the word involved. One of several examples from the song is:
Well, I see you there with the rose in your teeth
One more thin gypsy thief
These lines also demonstrate how Cohen carefully weaves together the sounds of different words in his song. Beyond the simple rhyme, we also have a case of alliteration (similar sounding first syllables) in thin gypsy thief, both of which pick up on the last syllable of the line above: teeth. To go back to the first line, we have two examples of assonance (similar vowel sounds). The two pairs are both in italics: "It's four in the morning, the end of December."
Aside from the well-crafted sounds of the lines, many of the sections are also drenched in ambiguity, wavering between literal and figurative interpretations. As an obvious first example, when Cohen sings, "New York is cold," we don't know whether he's referring to the obvious fact that it's freezing outside or instead to the fact that his relations with the people around him (Jane included) are cold in nature, hostile and distant.
The title offers another example. A literal "blue raincoat" takes on symbolic power; it gestures toward a time of youthful promise, when fame seemed just around the corner.