This amusing line is delivered by Simon with a completely straight face.
Simon, who is at this point in his career going through a midlife crisis of sorts, asks himself why he is so "soft in the middle" - a nice way of calling himself both weak and fat. The image is strengthened by the "beer belly" he describes later in the same verse.
Following the failure of Simon's 1983 album, Hearts and Bones, his career really was in need of a shot at redemption by the time of Graceland's release.
By 1984, Paul Simon had fallen far from the spotlight. The glory days of Simon & Garfunkel were fading into distant memory, and at times he had even lost inspiration to write new music.
Here Simon describes his feelings upon arriving for the first time in South Africa.
The line captures the typical reaction of an American first setting foot in the Third World -feeling slightly out of place yet simultaneously entranced by the unfamiliar sights and sounds.
In this specific case, the language he doesn't speak is most likely Zulu.
At the time Paul Simon wrote "You Can Call Me Al," South Africa had only two official languages: English and Afrikaans, the Dutch-derived language of the white Boer community. The African languages spoken as first tongues by the majority of the population - including Zulu, the country's most widely-spoken language - received no official recognition from the apartheid-era state. Since 1994, however, the post-apartheid South African government has adopted 11 official languages: English, Afrikaans, Zulu, Xhosa, Tswana, Sotho, Sepedi, Tsonga, Swati, Ndebele, and Venda. This is intended to serve as a testament to South Africa's new self-identity as the "rainbow nation."