The story is told in a very fatalistic manner. The narrator constantly speaks as if things could not have happened any differently than they did. Thus, it is not surprising that the "point of no return" comes much sooner than one would expect. Given the historical background and the Captain's fear of mutiny on his ship, it's clear that after Claggart accuses Billy something horrible is going to happen, even if it's not clear what.
Vere is perplexed after Claggart comes to him and accuses Billy Budd. He resolves to invite Claggart and Billy into his cabin in order to discuss the matter privately. That said, neither Vere nor the reader has any idea how things are going to turn out. It seems as if Vere thinks that Claggart will back down when he has to accuse Billy to his face. Though tragedy seems certain, a resolution is nowhere near.
After Billy hits Claggart, Vere almost immediately announces that he will have to hang, no matter how unjust it is. From that point on, the story plods toward its inevitable conclusion, but it is quite clear what that conclusion will be.