At Sea and Aboard a Whaling Ship, 19th Century
This is where we first meet Manjiro and the other Japanese fishermen, and even though most of them hate the whole lost-at-sea-away-from-home gig, Manjiro loves the sea and ship life because of all the sense of adventure and camaraderie among the crew.
Of course, camaraderie isn't easy to come by at first, but the crew eventually becomes a family of sorts for Manjiro:
They had all seemed so big and so hairy and so fierce when he'd first encountered them, but now he knew them as mostly kind and pleasant men. He had ceased to identify them by their skin or hair colors; now he knew them by their names and personalities. (2.7.6)
It's a motley and diverse crew, but just as Manjiro eventually comes to view each member has an individual instead of just understanding them based on their appearances, so, too, do they come to understand him as an individual.
On top of that, the sea and ship life (at least on the John Howland) represent freedom and potential. Manjiro tells Captain Whitfield that, to him, "'Future is like ocean… Big mystery, many danger, much beautiful" (2.7.37). Gee, sounds like a good setting for a story about a young man on the adventure of a lifetime, now doesn't it?
Fairhaven, Connecticut, 19th Century
Fairhaven is the small, American town that the Captain takes Manjiro to. For the Captain, it's like family; in fact, when he and Manjiro arrive at his house, which is closed and abandoned, it's his neighbors the Akens who take them in, having them stay in their home until the two men get settled.
At the same time, because Fairhaven is small, it's also—like other places—wary of strangers, especially foreigners like Manjiro. Because of Fairhaven's xenophobia, the Whitfields bounce from church to church until they can find one that more than just tolerates Manjiro, but accepts him whole-heartedly.
In this exclusivity, Fairhaven (and America in general) may share more similarities with the isolated Japan than we might expect.
Japan, 19th Century
We're not giving a specific place within Japan because the different settings Manjiro ends up in all go back to the same idea: Japan is way too isolated and narrow in its thinking. That's because it's still the Edo period (the Meiji period is what brings Japan into the modern, industrial era).
The fact that Manjiro gets imprisoned twice in Japan on suspicion of being a spy just because he was a castaway and lived in America for a while should give you an idea of just how isolated and anti-foreigner Japan is during the era our book is set in.
That said, there is a small exception: Manjiro's hometown. Sure, the Epilogue tells us that Manjiro never really left that cloud of suspicion behind him while he lived in Japan, but the way his hometown greets him indicates that—sometimes—a small place like his hometown can be just as supportive as a small American town like Fairhaven.
Here's a brief description of what the neighbors do to welcome him: "Later, after neighbors arrived, bringing red sea bream mixed with boiled rice, red beans, warmed sake, and other gifts of food" (5.40.28). The custom of bringing food to Manjiro is not unlike the way the Akens open their door to Captain Whitfield and Manjiro after their period at sea.
Small towns—even ones that are scared of strangers like Manjiro's hometown in Japan and Fairhaven in America—may not be as unsupportive of the new and different as we might think.