When the Present has latched its postern behind my tremulous stay (1)
This is kind of a weird, unexpected way to open a poem: the "Present" moment (which hasn't actually happened yet at the time the poem was written) shuts the back door ("postern") of life on the speaker. It's as though the "Present" moment of time has it in for the speaker: we can just imagine the "Present" kicking the speaker out, yelling, "don't let the "postern" door hit you on the way out!" Maybe the past was kinder to the speaker?
[…] will the neighbours say, (3)
The speaker introduces one of the main ideas of the poem – the question of what his neighbors will say about him after he's dead.
"He was a man who used to notice such things" (4)
The speaker uses the present and the future tense throughout the first part of each stanza, and then switches to the past tense when quoting what he imagines the neighbors will say: "He was a man who used to notice such things." The "such things" seems odd, too, especially after this absolutely amazing scene from nature in which the month of May turns into a bird or a butterfly "flapping" its wings. The neighbors just sum all that up by calling it "such things"? Do they just not get it?