The speaker in "Piedra de sol" is totally obsessed with the past and his memories, even though he's a pretty forgetful guy. In fact, the whole poem is a battle he has inside of his own mind to try to remember and recover the lost past. And who can blame him—he traveled the world, spent time with lots of girls, and now he's an old man waiting to die. We'd probably want to revel in our memories, too.
Memory is all the speaker has, and that's why it is so important to him.
The speaker uses memory to escape the present and the future, because he's so unhappy.