We always want to be careful about identifying the speaker of the poem with the poet herself. But in this case, it really can't be avoided. Giovanni speaks of how she learned to love jazz, coffee, and hardcover books. She's speaking of a place and time with which we know she is factually connected: she spent summers with her grandparents in Knoxville and eventually moved there to attend high school. Grandma and her kisses are all here.
It's clear that the first-person speaker of this poem has Giovanni's experiences and memories. We also know that she writes from personal experience and includes herself in many poems. If you're still not convinced that Giovanni is sharing a genuine personal experience, that's okay and very reasonable—especially since writers are known to take "artistic license" with the most autobiographical of memories.
In fact, we can actually see Giovanni exercising such artistic freedom in lines 31-32: "Probably they said something humiliating since southern/ Whites like to humiliate southern blacks." Here, she's telling us that she doesn't really know if Mrs. Long faced discrimination at the "big library." She's just hedging her bets that the librarian did, based on the norm.
Still, we have to consider the purpose of the poem. Since it is a poem of admiration and celebration (yay, Mrs. Long!), we feel justified in saying that the poet is also the speaker in this poem. Giovanni is stepping into the first-person position to give a well-deserved shout-out to an influential figure in her early life.