Joe: I... want... to touch you. Can I please just touch you... um, here?Stage Direction: He puts his hand on one side of Louis's face. He holds it there.Joe: I'm going to hell for doing this (3.6.23).
When Joe goes home with Prior, it could be seen as the ultimate betrayal of his wife and the beliefs that he's held to his entire life. It could also be seen as the first time in his life that he's not betraying himself.