The last line alone is enough to tell us that "Portrait in Georgia" is very much about race. But it's not as simple as including the words "black" and "white." The politics at work in this poem are difficult to suss out at best, and impossible the rest of the time. But we think that's totally the point. Even in this poem's most mysterious moments, Toomer is challenging the very notions of race on which our readings of this poem are founded.
This poem is historical, and describes a fictional lynching of a black man, because of something this white woman said. In other words, the two figures in this poem knew each other in this poem's world.
The body of the white woman and the body of the black man mix to create the body of the poem. They're all one thing here.