He's mysterious, he's older, he'd rather pessimistic, he's "seen things," he's been changed... this, friends, is the weary traveler, a famous figure (always a man, we're afraid) who appears in tales of long voyages. It all starts with Homer's Odyssey and continues to be an image of the life-changing and exhausting toll of long trips to faraway places. Raphael Hythloday is just such a man:
"I was about to return to my quarters, when I happened to see [Peter Giles] talking with a stranger, a man of quite advanced years. The stranger had a sunburned face, a long beard and a cloak hanging loosely from his shoulders." (1.9)
We can think about the unique look of this figure as a way of signaling on the outside a change he's undergone on the inside. He doesn't look like most people because he's been through things most people haven't. The realities of long-distance travel by ship were harsh, and many explorers returned to their native lands physically and emotionally changed; these real-life adventurers actually did resemble their fictional counterparts.
If you're intrigued by these mysterious travelers check out Samuel Taylor Coleridge's poem Rime of the Ancient Mariner for some more juice.