partly because of my love for you, partly because of your love for yoghurt (4)
This is a pretty casual expression of love, don't you think? Usually there's a choir in the background, some gently lit snowflakes, or, heck, even a lone violin. But in this case we get… yoghurt? This gives us an idea of the casual, conversational tone of the poem to follow. The speaker is just totally head-over-heels for the you, but he's also trying to avoid going too far over the top. He follows up his declaration of love—a potentially vulnerable moment—with an attempt to equate it with the addressee's love of yoghurt. This lets us know that we're not in for your typical lovey-dovey romance.
partly because of the secrecy our smiles take on before people and statuary (6)
One of the great things about love is the privacy it creates. When you're in love with someone, it's like you live in a world with just the two of you. You have all these inside jokes and secret looks, and they are only meant for that other person. We get the sense here that the speaker and the "you" of the poem are a kind of exclusive club (with lots of velvet ropes and really big doorman to keep out the rest of the world).
in the warm New York 4 o'clock light we are drifting back and forth (9)
This sounds a little trippy. The speaker's not talking about astral projection, though. He's talking about that floating-on-a-cloud feeling that comes when you're in love. Notice, too, that the couple is "drifting back and forth." They are leaving themselves (though not literally, we hope) and moving toward each other. You know that stage of love where you sort of… become the person you are in love with? Or at least feel closer to them than you thought possible? This is what our speaker is on about here.
I look at you and I would rather look at you than all the portraits in the world (13-14)
Things that make us say, "Awwww." That's just the most heartwarming thing we can think of to say to anyone. Who wouldn't want to hear that they matter more than all the great art in the world? Now that's showing the love.
and the fact that you move so beautifully more or less takes care of Futurism (17)
Not only does our speaker take the addressee over every portrait in the world, he says that you move better than any school of art could hope to capture. Futurism was a movement of artists that were all about motion, excitement, speed, and energy. And yet, they can't even come close to watching this person move. Here we see how love brings our speaker back to favor reality, or at least love-reality, over art.
it seems they were all cheated of some marvelous experience which is not going to go wasted on me which is why I am telling you about it (24-25)
These final lines are really profound, when you think about it. Put it this way: when you ride a roller coaster, are you aware of how much fun it is to ride a roller coaster (for some of us, anyway)? Probably not. You're probably too busy actually having the fun to think about how great it is to have fun. The same thing is going on for our speaker. He is acutely aware of how great it is to have someone to love, and he's not about to take that for granted. Instead, he's embracing the experience of love as a supreme moment in his life (where most of us would simply just enjoy the moment, and then look back on it later and sigh wistfully). That's the realization that he's sharing with the addressee of the poem, and with us, too.