Want to come up with a good way to make your verse seem like it's drawn from antiquity (like, say, Roman antiquity) when it's also firmly lodged in the twentieth century? One way to do it is to play with the sorts of word orders that poets from antiquity used to use. See, back in the old days, when English was primarily an oral language, word order wasn't as standardized as it is now—and Cummings uses that fact to create an allusion to olden times, as well as an inventive new structure for the poetry of this century.
As we said back in the summary, the way that Cummings treats love as the stuff of legend is pretty cool. For one thing, it divorces love from any of the everyday-ness that tends to occupy our lives. Love isn't the stuff of the office or school or any particular place and time. It's for the ages. And inverting sentence structure helps convey the mythic structure of love itself.
Line 1: "Went my love riding": subject and verb get inverted here, which gives us a sense that the poem is from the days of yore, when poets often inverted syntax for dramatic effect.
Line 6: "Be they" is a rather funny way of saying "they are." But by not conjugating the verb "to be," Cummings seems to fix the image of the running deer into a permanent image. They're almost commanded to be fleeter, in a sense.
Line 11: This line repeats the same structure of the first stanza. In fact, it creates a complete repetition of the first line's subject and verb. Doing so makes the poem focus intently on the fact that riding is, well, pretty much everything that is going on in the poem.
Line 26: Notice how the subject and verb are inverted again? By the end of the poem, we're starting to notice a pretty set pattern.