Douglass is born on a huge slave plantation in rural Maryland, one of hundreds of slaves. When we think of slavery, we usually conjure up an image of the Deep South, or the "Old South," places like Georgia or Alabama. Maryland was one of the northernmost states where slavery was legal (and close to Pennsylvania, a free state), and life there wasn't quite as bad for slaves as it was in the Deep South. In fact, in the letter to Douglass that introduces the book, Wendell Phillips says that slavery "appears with its fairest features" in Maryland. Phillips' point, though, is that if slavery in Maryland is as good as it gets, it's still pretty bad. Douglass doesn't want us to think of slavery as a thing that can be better or worse – all slavery is wrong.
Douglass spends much of his young adult life in Baltimore, and his situation is better there than on the plantation. He is legally enslaved, but in many ways he can almost live as though he were free. And when he eventually escapes to New York, his situation is better still. But even in the North, he isn't really free, since he always runs the risk of being kidnapped and brought back down south. He moves farther north to Bedford, Massachusetts, as soon as he can, but one reason he joins the abolitionist movement is that he can never truly be safe while slavery exists anywhere in the United States.