The house feels even more isolated than it really is—thanks to the wind—which makes it seem as though the house were alone on the ocean.
At noon I scaled along the house-side as far as The coal-house door. Once I looked up— Through the brunt wind that dented the balls of my eyes (9-11)
Our speaker, even though he later mentions a "we," seems here to be going through this ordeal alone. His experience with the wind is his alone, with the emphasis on being alone. That's enough to dent anyone's eyeballs.
[…] Now deep In chairs, in front of the great fire, we grip Our hearts and cannot entertain book, thought Or each other. (18-21)
The wind breaks down human relations. It imposes itself on everyone's attention and removes awareness of everything else, including other people.