Why linger, why turn back, why shrink, my Heart? Thy hopes are gone before: from all things here They have departed; thou shouldst now depart! A light is pass'd from the revolving year, And man, and woman; and what still is dear Attracts to crush, repels to make thee wither. The soft sky smiles, the low wind whispers near: 'Tis Adonais calls! oh, hasten thither, No more let Life divide what Death can join together.
Now we know the speaker's talking to himself. He wonders if his heart is strong enough to meet Adonais-Keats in the afterlife.
After all, he thinks, his hopes are all gone, now that the youth is dead. The "light" of his hopes has passed from humanity, and the things dear to him are withering up and dying. Yeah, this dude sounds pretty hopeless.
He even imagines that the sky and the wind are whispering to him, telling the speaker that Adonais-Keats is calling for him to join him in the afterlife. He figures that he shouldn't let life divide them, and that death can bring them together.
It looks like his grief has brought him pretty low.