partly because in your orange shirt you look like a better happier St. Sebastian (3)
What? Saints in orange shirts? It may seem like a stretch to think that simply putting on a t-shirt can turn one into a beatific vision (unless it's t-shirt that glows softly and plays angelic music). But if things look different, and wildly better, to people who are in love, well then anything's possible.
partly because of the fluorescent orange tulips around the birches (5)
Okay. You say that the tulips are glowing like light bulbs. We're going to ask that you refrain from operating any heavy machinery for a while, please. What's that? You're in love? Oh! Why didn't you say so? Of course, the flowers are glowing. The whole world is aglow when you're in love.
in the warm New York 4 o'clock light we are drifting back and forth between each other like a tree breathing through its spectacles (9-10)
We love these lines. They're so odd, serene, and, well… poetic. Sure, it may sound crazy to talk about drifting back and forth out of your body like some kind of blissed-out Casper the Friendly Ghost. And the trees who happen to be wearing scuba masks aren't helping things make much sense. But that's love, folks. It's a wonderful sensation where you get to escape yourself for a while, and it has very little to do with logical reality.
and the portrait show seems to have no faces in it at all, just paint (11)
Our speaker has been shown a new reality by love, one in which art has ceased to be art for him. Instead, it's become "just paint." In actual fact, all portraits—on one level of reality—are just paint. The skill of the artist, however, creates a new reality for the viewer. The trouble is, our speaker's found an even better reality with love, and so the skills of the artists are lost on him. That's a real shame. Or is it?
it seems they were all cheated of some marvelous experience which is not going to go wasted on me which is why I am telling you about it (24-25)
In these concluding lines, the speaker delivers the heart of his message. Love has awoken him to the bliss of his current reality (i.e., being in love). He's not going to go around anymore, trying to escape his reality by admiring other versions of reality, namely art. Oh no. He's going to appreciate the reality of his existence instead, which to him is far more enjoyable than any artistic substitute… except for maybe "The Polish Rider." Maybe.