Nil sapientiæ odiosus acumine nimio.—Seneca
Nothing is more hateful to the sense than too much cunning.
Seneca is the street name of Lucius Annaeus Seneca, The Younger (meaning, junior), an influential philosopher who lived in ancient Rome. Seneca was a Stoic. Stoics were a group of Greek—and then later, Roman—philosophers who believed that the goal of life was for people to stop being such big babies about everything.
Okay, not exactly. But they did believe that you needed to cultivate some self-control and objectivity.
Anyway, in plain English the phrase quoted here means something like: "Too much cleverness doesn't make good sense," or "Don't be too smart for your own good." Set at the beginning of the story like this, it sounds almost like the moral that tells us what kind of message to take from the story.
But it also might be directed at a specific character. Let's check out the options:
Dupin tells the narrator that G—'s success in detecting things "depends […] not at all upon the acumen, but altogether upon the mere care, patience, and determination of the seekers" (98). So, at least from Dupin's point of view, the epigraph can't apply to G—.
Dupin would most likely think that the quote applies to D—. D— is the "unprincipled man of genius" whose extreme cleverness has caused him to lose his good sense (120), since he's so taken with the cleverness of his theft and hiding place that he's overlooking the phenomenally stupid action of blackmailing a royal lady.
For that matter, couldn't the phrase apply to Dupin himself? After all, he's the (seemingly) cleverest guy in the story. He's making copies of seals out of bread dough, spying from behind his green glasses, dreaming up plots, and even staging deceptions. And maybe he ends up going too far. In leaving his revenge note for D—, he's showing off his cleverness in a way that, well, doesn't seem too sensible.