Yes, despite the sex we’re sure Hester Prynne and the Reverend Dimmesdale had, it is all in the past as far as this book goes. None of it occurs on these pages. Instead, get ready for lots of guilt, shame, and suffering because of the sex that happened some time in the past.
However, the novel was probably racy in its day, given the subject matter. We can see the National Examiner headlines now: "Young, sexy married women has child with young, unmarried minister and plans to run away with him after her estranged husband is transformed into an evil, demonic person seeking revenge on his wife’s unnamed lover!" Yes, some definite potential there.