The speaker describes seeing fifty-nine swans swimming at twilight on a gorgeous October evening. Apparently, he's been watching these swans for quite a while—years, even. And while he's watching, they all take flight in a big flurry-hurry.
As it turns out, this kind of breaks this poor young guy's heart. That's because his life has changed so much since the first time he saw these swans. The swans on the other hand, haven't changed much at all. They haven't grown old, and they still go where they please. But in his heart, the speaker knows these swans won't be around forever, and he wonders where exactly they'll be hangin' when he wakes up to find that they've finally ditched him for greener pastures (or clearer waters, as the case may be).