Now, Claggart's a pretty good-looking guy, minus a slightly over-large chin (think Jay Leno). But he can't hold a candle to the handsomeness of Billy Budd. In particular, what distinguishes them is that Billy has a ruddy (rosy) face, whereas Claggart's is pale and haggard (old-looking).
Maybe, when Billy dropped his soup and Claggart let slip his mean-spirited little joke, he accidentally revealed what it was about Billy that made him hate him so much: his handsomeness.
The problem is that no one ever confesses to envy because it's such a petty, embarrassing sin, and envy of someone else's good looks seems particularly trifling. Claggart is no exception.
Intelligent as he is, his problem with Billy is perhaps that he has a "disdain of innocence – to be nothing more than innocent!" (12.3).
The narrator presumes that, given the fact that there is some elemental evil in Claggart, there is nothing for him to do but to play out his allotted part and to let that evil do what it will. Pretty fateful view of things, yeah?