According to 404 Media, Amazon is purchasing a large number of rare books, cutting off the spines, and scanning them for AI training. 404 Media installed a tracking device on the rare book, which ended up arriving at an Amazon facility in Las Vegas.
The facility, known as VGT3, identifies itself with the symbol of a dinosaur holding a book in its claws. “We purchase books through commercial channels to improve the products and services our customers use,” Amazon told 404 Media in a statement.
Companies like Amazon require an unfathomable amount of text to train their LLMs. LLMs are already pulling in as much information as they can from the internet (and, in the case of Anthropic, illegally pirated books). Rare books, especially those that are out of print or impossible to find on the Internet, provide a new source of much-needed training data.
These texts are particularly valuable because anything published before 2022 is unlikely to have been written by an LLM. When an LLM is trained on AI-generated text, “model collapse” can occur if the quality of the LLM’s output decreases after ingesting a large amount of AI-generated text.
