There’s a difference between pre-training and reinforcement learning.

Pre-training is when the models are trained on scraped data. The models basically learn how to parrot the Internet. Usually they have a cutoff point to avoid training on slop.

Reinforcement learning is where most of the training is happening. They’re given tasks (for example, make a calculator app), and then judged by how well they complete the tasks. This is more resistant against model collapse because they’re not judged by how well they parrot slop.

As part of the reinforcement learning process they give the model access to tools like searching the internet so the newest programming language features can be accessed while the tasks are performed. This is where OpenAI is shifting to a cached version of the Internet the models can interact with instead.

Unlikely. They’ll need to train the models so they can perform their tasks even if the internet is full of slop.

There’s so much AI slop out there that the frontier labs risk model collapse if they continue to train on it.

That’s why they’re resorting more and more on reinforcement learning. Give the AI a task. Quantify how well the AI performed the task. Less parroting what’s said on the Internet.

They’ll still scrape the internet. The difference is that the models will only have access to a cached version of the internet during the reinforcement learning process (for example, check the documentation of a programming framework). I’m surprised it took them this long to realize it’s a bad idea to do reinforcement learning with live internet access.

Translation: they’re still going to scrape internet. They’re just no longer going to perform reinforcement learning with access to live internet (instead they’ll use a cached version of the internet).

They’re going to use AI to install the AI.

Recursive Self Installation

Wordpress is still commonly used for e-commerce. You’re probably visiting more PHP sites than you realize.

There might be “better” languages, but PHP is still used by ~70% of the web.

AI is about making machines that mimic human level intelligence. Whether they possess actual intelligence doesn’t matter. They just need to act intelligently.

You can disagree with the terminology, but it has been an active field of study since the 1950s.