Nautilus Island's hermit (line 1)
This line isn't about our speaker, but reflects the isolation of the townspeople. The old lady stays in alone all the time.
the hierarchic privacy (line 8)
Again, Lowell reinforces that this a place where people like to keep to themselves. It doesn't exactly give us a warm and fuzzy feeling about the town.
we've lost our summer millionaire, (line 14)
OK, so this isn't the most serious part of the poem, but it gives us the sense that the population decreases on the island during the off-season, and that it gets a little lonelier.