The little horse must think it queer To stop without a farmhouse near (5-6)
How does the little horse know there isn't a farmhouse near, and, more importantly, why does the speaker choose to stop so far away from any other humans. The farmhouse represents something different than the village does – it's a bit more nature-loving than the village. But the one thing that both farmhouse and village have in common is people, and that's the one thing that our speaker doesn't seem to like too much.
But I have promises to keep, (14)
The word "promises" is an interesting one. Promises usually involve other people, and they usually involve the future (whether immediate or distant). In this way our speaker seems to be choosing people and his future over nature and the present.