The two major episodes of Lord Jim deal with different kinds of "others." Aboard the Patna we get lower-class men contrasted with Jim, who is a middle-class gentleman. On Patusan, we get native islanders contrasted to Jim, the white imperialist. What's interesting is how Conrad flips expectations around in both episodes. Jim is never the one with all the power, even though his racial, economic, and national status would seem to empower him. And unlike a lot of authors of the era, Conrad uses "others" to point out negatives in the dominant group (white Englishmen) rather than write off all "others" as negative.
The native characters on Patusan are cast in a fairly positive light through Jim and Marlow's descriptions. They aren't cast as negative "others."
The non-white characters are consistently described in negative terms throughout the novel, revealing Conrad's racist tendencies.