Jude seems like a stand-up guy when he marries Nel. He wants more than anything to work on the new bridge so that he can be part of something lasting. For a long time, he is a responsible husband who takes care of and loves his wife. He and Nel seem to have a happy marriage, despite the fact that Jude is consistently frustrated in his attempts to get the work he wants because of his race.
But when Sula comes along, all of this changes. Perhaps he sleeps with Sula because he is so frustrated with the "hard row [he has] to hoe" (1937.174) as a black man. Perhaps he just needs something to shake his life up. Regardless of the reason, he ruins his marriage, fails to keep in contact with his kids, and destroys for many years the close relationship between Nel and Sula. Some critics have suggested that Sula offers a critique of institutions such as marriage (Barbara Smith suggested a lesbian reading of the novel in part because of this very thing), and it's possible that Jude is one of the vehicles through which we get this critique.