Nothing lasts forever in this poem, and no one is perfect. In fact, Auden is known for being anti-heroic in all of his poems. He doesn't aggrandize people (build them up) and make them all-powerful or magnificent. Instead, he always acknowledges people's imperfections and flaws. For Auden, though, these flaws are the best part of us. Our flaws are what make us human.
Line 2: The speaker references both his own imperfection and his beloved's here. He is without faith, and his lover is "human." This word means that he's a human being, of course, but it also has other connotations: if you are human, you aren't perfect, and you won't live forever.
Lines 9-10: The speaker says that his beloved is "mortal, guilty but to me / The entirely beautiful." The speaker doesn't resent his lover's flaws; instead, he loves them. Instead of building his beloved into a perfect human being, he acknowledges that he loves his weaknesses as much as his strengths.
Line 14: The vision of the lovers here isn't extraordinary; in fact it is "ordinary." Theirs is not a special love. It's a love like everyone else's: sometimes beautiful, sometimes ugly.
Line 36: The speaker asks his lover to "find the mortal world enough." He doesn't want to shoot for the stars or imagine the heavens. He wants them both to be satisfied with the imperfect but beautiful life that they have together on earth.