Frederick Douglass's friends in the abolitionist movement were all extremely faithful Christians, but Douglass has some really harsh criticisms for slave owners who claim to be Christians. (Douglass believes that a person can't both be a Christian and a slave owner.) Not only does Douglass hate hypocrites, but he also tells us that religious slave owners are even worse than those who don't pretend to be religious. This sometimes got Douglass in trouble with Christians who thought he was attacking them instead of religious imposters. (That's why he wrote an entire appendix just to explain that he was against religious hypocrisy, not religion itself.)
For Douglass, religion isn't about what you do on Sunday; it's what you do the rest of the week that counts. Slaveholders who go to church, he believes, are not real Christians.