the sweet flesh would turn sour. (line 21)
He may as well be talking about his aging lover. Sweet flesh turning sour, or young flesh getting wrinkly, is the reason so many cosmetic companies are in business! Humans are constantly trying to preserve themselves and slow the effects of aging. But we all know there's no stopping it.
Each year I hoped they'd keep, knew they would not. (line 24)
Bummer. This is the age-old desire to stay young and alive. This is why so many fairy tales and myths have some version of the quest for the fountain of youth. Death is not the easiest thing for humans to reconcile. Regardless of how often we're told we're going to die, we try best we can to stretch out life.