"The Day Lady Died" is a simple description of the poem, which gives a blow-by-blow account of the speaker's activities on the day Billie Holiday died. "Here's where he went. Here's what he did." But wait, there must be something more, right? The title contains Holiday's nickname – "Lady Day" – backwards. It's like a secret code. But the connection only works if the reader knows that Holiday's nickname was "Lady Day." Otherwise you wouldn't even know that the poem is about Holiday. You might think it was about Lady and the Tramp or something.
O'Hara liked to write poems he could read aloud to his friends, and he knew his friends were cool cats who knew and followed the jazz scene (source). We can imagine a modern reader getting a bit irritated at his assumption of such cultural knowledge, but that's just how O'Hara rolls. By reading the poem, you become a member of the club.