The novel is peppered with frequent allusions to different parts of the Bible. The most obvious is the reference to Genesis 30:1-3 (Epigraph), with its catchy phrase, "Give me children or else I die." That text, with its focus on bringing a "maid" or Handmaid into a childless marriage to create heirs, is the fundamental idea behind the Republic of Gilead. (For more on this, see "What's Up With the Epigraph?" or "Shout Outs.") Specific parts of the Bible that glorify marriage, that absolve men of adultery for the purposes of childbirth, and that convict women of it, have been cherry picked from the text and made into law. Other parts, such as the ones that emphasize meekness and humility, have been used to dictate behavior to the Handmaids. (It goes without saying that there's only one authorized religion permitted in Gilead, the one promoted by the state.)
Obviously a variety of Western monotheistic religions rely on versions of the Bible in different ways, and the Bible is often seen as a moral weight in society. But in the United States, Biblical law is separate from state and federal law. You don't have to abide by what your neighbor says is in the Bible if you have a different interpretation of it (and vice versa). In Gilead, what the government has decided should be taken from the Bible has become absolute law. The authority the Bible already has becomes even more powerful.
Strange, small pieces of Biblical text show up frequently throughout the book. This is particularly evident in place names and propaganda. For example, there's Gilead itself. Within it, all the stores the Handmaids are allowed to shop at have Biblical names: Loaves and Fishes, Milk and Honey, All Flesh, Lilies. The hotel where the prostitutes are kept is called Jezebel's. Whenever the narrator remembers a piece of dialogue or something that happened at the Center, it usually includes a piece of Biblical content. These references, though, have been bastardized or altered in some cases to further the goals of the Republic, so even if knowledge of the Bible is excellent, you might not catch them all.