Screenwriter Christopher McQuarrie and director Bryan Singer knew each other in high school in Princeton, NJ. (No word on whether they exchanged BFF bracelets). Singer was trying to become a filmmaker, creating his own home movies, while McQuarrie wanted to become a…wait for it…writer.
But their paths to these goals ran along different lines.
While Singer stuck to a straightforward road, studying filmmaking at NYU and USC, McQuarrie worked at a grammar school in Australia, got fired, hitchhiked around, and then worked at a detective agency, which—according to him—was really more like a security guard agency. After that, he almost joined the NYPD, before his screenwriting connection with Singer finally paid off. (Source)
He was "finding himself"—like lots of people in their early twenties. Finally, he moved to California to write screenplays. When he finally did find himself, it turned out that he was a screenwriter who wrote hit movies and made a ton of money. Which: not exactly a bad version of yourself to find.
He wrote Singer's first film, Public Access, which killed at the 1993 Sundance Film Festival. After this initial success, they decided to team up again, basing their script of a simple idea McQuarrie had for the movie poster: a bunch of criminals standing in a police lineup. He thought the title would be The Usual Suspects, inspired by a column he'd read in Spy magazine…which in turn, of course, is inspired by one of the most famous quotes in the movie Casablanca.
That was the genesis, but clearly, there needed to be a lot more to the movie than that… The story-constructing duties fell to McQuarrie. (Source)
In crafting his script, McQuarrie actually based the names of the characters on people he was working with at an LA law firm (he still needed a day job at the time). For instance, Dave Kujan was a real guy and so was Fred Fenster.
One day he met a lawyer named Keyser Sume—McQuarrie changed it a little, apparently for legal reasons, to Keyser Söze. (Of course, now that this story is public knowledge, people probably assume this lawyer actually was a criminal mastermind. Cloaking device: failed.) (Source)
Also, McQuarrie got the idea for the movie's big twist when he was looking at bulletin board in his office and noticed it was made by Quartet, a company based in Skokie, Illinois—just like the bulletin board in the movie, which reveals the truth to Special Agent Kujan. (Source)
So, basically: we owe one of the greatest surprise endings in movie history to the label on a bulletin board.