Sometimes they attract, sometimes they don't, but more often than not they tend to point at the way the world works. Our lives and the universe are all about opposites. We can't escape them because they're part of what we experience every day. In Ashbery's poem, they work the same way. In order to understand one thing, you first have to know what's on the other side of the fence. And even then, no one's making any promises that there will be a neat little explanation on the other side.
Lines 5-6: It doesn't matter how far from the world you'd like to be. Either way you cut it, you're still part of it and will, in that aspect, somehow "agree with it."
Lines 15-16: The silence has noises and the canvas will have something emerge from it. There's always "stuff" to everything. So before we revel in silence and emptiness, we have to understand that its opposite is in there somewhere.
Line 18: How can something be "placed" somewhere and also be "moving"? Just like everything else, to understand one is to understand its opposite. And all these opposites are what contribute to its "puzzling" nature. But don't worry; it's all good because this is just the way the world (and us in it) works.