They that have pow'r to hurt, and will do none, (1)
If the first thing we learn in the poem is that it is talking about a group of people ("They"), the next thing we learn is that they "have pow'r." Their power takes a particularly disturbing form: it is the power to hurt. And yet, before the first line is even finished, this blow gets softened; we also learn that they "will do none," that is, that they won't hurt anybody. Is this ability to refrain from doing everything that they have power to do just another form of power?
Who moving others are themselves as stone, (3)
Ice man alert. Apparently one of the things that make these folks so powerful is that they can move people. This line makes us think of powerful leaders, persuading entire countries with speeches that are calculated, but convincing. Often, that gets us into some hot water, so this line isn't exactly all roses.
Unmovèd, cold, and to temptation slow— (4)
This line basically expands on the idea from the end of line 3, that the powerful people are "cold." We get some extra description about the creepy coldness of the powerful people, and maybe a little bit of a hint about how they are "moving others" in line 3. We can't be 100% certain, but it seems likely that the "temptation" the speaker is referring to is sexual temptation. This suggests that the powerful people might use sexual temptation to influence others, even as they themselves are able to resist such temptation (they are "to temptation slow"). Hey, it is a sonnet, guys.