The title is a reference to the gospel song "Go Tell It On the Mountain," a popular Christmas carol because its lyrics refer to Jesus Christ's birth. The musical reference reflects the constant melodies that are sprinkled throughout this novel. Hymns are always breaking into the narration, reminding us that the novel takes place during a (very long!) church service.
The song's message is that the Christian message ("that Jesus Christ is born") should be spread everywhere, shouted from the mountaintops. This novel can be interpreted as Baldwin's way of telling his own experience of Christianity—with all its difficulties, bumps, and bruises—as well as his own coming-of-age.
Finally, we can't forget the image of the mountaintop (see the "Symbols" section for a lot more on this) that the title reflects. Throughout the novel, characters are portrayed as climbing a mountain to holiness. It's a tough climb, but the rewards at the top are more than just a great view.