The speaker is both a lover and poet. Not only that, but he's a big believer in the value of both. Throughout the poem he bemoans the lack of respect many people have for the work that goes into both writing and wooing the object of their desire. Love and poetry deserve respect, he believes, and he's sad to see that respect vanishing. He's pretty old-school; he likes his poetry well-crafted and his love affairs full of quotes. We can guess that he's well-educated in both topics, and likes to talk about them.
We know a bit about his personal life, too. The speaker loves, or at least once loved, one of the women he is speaking to in the poem. It doesn't seem to have gone well (much like Yeats' decades-long love for the object of his desire, an Irish lass named Maud). We get this sense because, when the topic of conversation turns to love, he becomes silent and sad, musing about his heart's weariness. Poor guy.