Apparently, the Party despises littering. Really, you just never know what kind of Party-defying thing could be lying about, so Oceanians are to deposit every scrap of paper they find into the ominously named "memory holes," slits in and alongside buildings leading to underground furnaces where all documents are destroyed.
Winston's job consists of faking the truth. Seriously. He alters print sources to ensure that they are in agreement with the Party's version of past and present events. Sometimes, this includes tracking down and forever erasing references to certain persons from historical documents once the Party has "vaporized" such persons.
Other times, this means cross-referencing made-up events in other sources so that the end product is more believable. And he wonders why he doubts the party's "history."
No, seriously, he wonders. And then he concludes that the Party's version of the past might not be true. He muses that once the act of forgery is forgotten, anything forged might seem as authentic as reality.