Like his friends and fellow writers in the Beat generation and the New York School, Frank O'Hara was all about capturing the tone of everyday, spoken speech in his writing. The idea was pretty radical at the time, actually, considering that poetry (in America, anyway) was previously held to be a place where Grand Ideas went to be expressed in all their complex glory.
"Eh, not so much," said O'Hara and his buddies. Like the English Romantics before them, O'Hara and co. strove to bring poetry to the masses, to ground it in common experience and language. That's one reason that "Having a Coke with You" is not called "Having Some Fine Champagne with You." It's also why you'll see words like "anyway," and only find two (count 'em) commas (aside from the list in line 1) in the whole poem. O'Hara mimics the speed and urgency of an intimate conversation. (For more on this style of writing, check out "Form and Meter.")
Don't let that fool, you, though. Just because it's approachable, doesn't mean that our guy doesn't have something to say. Specifically, O'Hara was interested in art, and its relation to reality and our lived existence. As both an art reviewer and an assistant curator at New York's Museum of Modern Art, O'Hara thought and wrote a lot about art, and so it's not surprising to see sculptures and paintings interspersed throughout the lines in this poem.