Symbolism, Imagery, Allegory
Think of this as the Sisterhood of the Traveling Pants, except the pants is a dress, the sisterhood includes a dude, and the travelling we're talking about is time travel.
Vera Louise, Golden Gray's mom, was once a pretty young thing with a secret black boyfriend. This is a big no-no in the antebellum South. And when she gets together with her man, she wears a green dress so she can be disguised in the grass. When she has to leave town quickly, her boyfriend keeps her dress.
When the boyfriend—now called Hunter's Hunter—is all grown up, he's visited by his illegitimate son, who brings an unconscious pregnant girl into his house and covers her with a… green dress. Why does a hunter have a woman's dress in his house? It's the same dress his girlfriend wore oh so long ago.
The pregnant woman ends up stealing the green dress and keeping it for years, at least until her adult son comes and discovers it in her cave-house.
So does this dress just symbolize the passage of time? Nope, not that easy. It also symbolizes the lasting impact of history (the dress lasts) and the power of memory. Check out Hunter's Hunter's quote:
"Tell (Vera Louise) I'm waiting for her and to come on out. She'll know the place we meet at. And tell her to wear that green dress. The one make it hard to see her in the grass." (7.16)
Hunter still remembers his boyhood love, and he still remembers their meeting spot. He also remembers her green dress—of course he does, because he's held onto it for more than eighteen years. Other characters may not have mementos from the past, but they also hold onto memories for decades, and the dress is here to draw our attention to this pervasive practice in Jazz.