The poem takes place in several places in Ithaca; it starts by the hearth in Ulysses' palace or castle, then points to port, and then somehow ends up there. By the end of the poem, we think that Ulysses is standing next to his mariners by the ship.
Pretend that Ulysses is George Washington. He's just won the Revolutionary War and is now living in the president's house in Philadelphia (the City of Brotherly Love was briefly the nation's capital); even though the presidential crib wasn't as fancy then as it is now, it was still really nice. Now, pretend that George Washington got really bored with living in this gigantic house in Philly and decided to head to the nearest army base and get his old uniform back on, as if he were preparing to go back into battle. If the poem were called "George Washington," our first president would start out by speaking to us from his humongous living room and eventually take us to an army or naval base.