Researcher Documents 103B-Token Usenet Corpus (1980-2013) — One of Largest Private Pretraining Datasets
r/MachineLearning·medium signal
A machine learning researcher has spent years building and documenting a complete Usenet archive spanning 1980 to 2013, totaling 103 billion tokens. The corpus represents one of the larger privately-held pretraining corpora, covering 33 years of unfiltered human communication predating social media. Post hit 67 upvotes and 12 comments on r/MachineLearning, notable for its unique temporal coverage of early internet discourse.