There are actually three letters in "The Purloined Letter," though only one of them is purloined. The other two are used to purloin. The letter that gets purloined is the one that the mystery person S— sends to the royal lady. D— doesn't just make a grab for it, but "leaves his own letter—one of no importance" (28) in its place.
Dupin mirrors this trick when he leaves a fake copy of the purloined letter in D—'s apartment. The difference in the two purloinings is this: In the first one, the queen knows that D— is stealing the letter, but dares not stop him. In the second, D— is unaware that Dupin is stealing the letter.
Some major players in the literary word make much of this point, arguing that the letter is a symbol of different processes of vision and knowing, and also that it's a symbol of the process of desire. (Check out "In a Nutshell" for more about their famous feud.)
(1) Vision and Knowing. The letter is right in front of the royal man, but he doesn't see it, and doesn't know what's going on. Similarly, the letter is right in front of G—, but he can't see it, though he does know it's there. Finally, when Dupin purloins the letter from D—, D—doesn't see that Dupin is after the letter and doesn't know why Dupin is there.
Got that? In other words (to majorly simplify it), the letter symbolizes the way we sometimes have the hardest time seeing what's right in front of us.
(2) Desire. This one might be a little trickier. Think about a letter. Sure, you could argue that writing is always meant to be read to some degree, but a letter really amplifies that intention. The whole purpose of a letter is to be read.
In fact, if you want to anthropomorphize a little, you could say that a letter desires to be read. The entire purpose of the purloined letter's existence to be read by the royal lady. When it's separated from her, it can't fulfill its purpose, because the letter that D— leaves in its place when he steals it stands in its way. But the third letter, the one Dupin leaves in its place when he steals it from D—, fulfills that desire.
Of course, we never find out if the letter does end up fulfilling its desire or not. We last see it in the grubby palm of G—. The letter's unfulfilled desire to be read is mirrored by the reader's unfulfilled desire to have it be read.