The garden of the forking paths serves as the story's central metaphor. It's another way to describe a labyrinth, or an outdoor maze. Of course, the garden isn't the only labyrinth in the story – the garden is a sort of meta-metaphor (if that makes any sense – check out our discussion of this in "Symbolism, Imagery, Allegory"). The garden's forking paths are a metaphor for the infinite diverging narratives of Ts'ui Pen's novel, which is in itself an allegory of time.
The garden also shows up literally in the story. It's an idyllic space in which the protagonist spends his childhood, a place of seclusion for Ts'ui Pen, and a maze that the protagonist must walk through in order to arrive at Dr. Albert's library. Try this argument on for size: in all of these gardens, the sense of time seems to be lost. After all, doesn't Yu Tsun's nostalgia for the gardens of his childhood imply a certain longing for the past within the present? Doesn't the scholarly work that takes place in the gardens of Ts'ui Pen and Dr. Albert suggest that universal truth transcends time? And what about Yu Tsun's walk along the forking paths leading to Dr. Albert's house? He says he forgets his "destiny," and that the evening becomes seemingly "infinite" (20).