Take up the White Man's burden-- Ye dare not stoop to less-- Nor call too loud on Freedom To cloke your weariness;
These lines give us more cheerleading for the white men. They need to take up this challenge and not ask for any easy outs. The example the speaker gives here is to ask a personified version of Freedom (note the capital F) to hide or cover up ("cloke") fatigue. In short, our speaker's message is: don't ask for an easy escape from this tiring work. Are we having fun yet?
By all ye cry or whisper, By all ye leave or do, The silent, sullen peoples Shall weigh your gods and you.
The pressure's on, white men. The people whom they're supposed to be "helping" will be judging ("weigh[ing]") everything the white men do.
Notice here that the locals are once again described as "sullen," or sulky, as in line 7.
What's more, it's not just a clash of people here; it's a clash of spiritual beliefs as the locals are also judging the white men's "gods."
When our speaker puts it that way, this scenario feels more like an ominous collision of two very different ways of life. And, as history tells us, that's pretty much what it was.