Researchers from Oxford and Cambridge have highlighted a major problem for future AI models: model collapse.
As AI-generated text floods the internet, future models could increasingly train on content produced by previous models. The result? AI may gradually lose the rare, creative and unusual patterns found in genuine human data.
The researchers found that repeatedly training models on generated data can cause them to lose the “tails” of the original data distribution, essentially forgetting the weird, diverse examples that make human knowledge so rich.
And this isn’t limited to ChatGPT. The researchers showed the phenomenon can affect multiple types of generative models, suggesting it’s a broader problem with learning from recursively generated data.
AI companies depend heavily on massive amounts of web data. If the web becomes increasingly synthetic, authentic human-generated data could become one of the most valuable resources in AI.
Source
