On Friday, OpenAI said that it will train its models using cached copies and simulated versions of the internet instead of the live web because the models are too unpredictable to be trusted with live internet access.
The frontier lab announced its new policy two weeks after its rival, Anthropic, began limiting its newest and, in theory, most capable models from accessing the live internet. Researchers adopted these policies after months of incidents in which models pursuing training tasks broke into external systems in search of answers, collaborated on obscure message boards, and developed exotic hacking strategies.
so was it the AI poison, or the pollution from ai warping data, like the 10th photo copy in a chain rendering the image unrecognizable
Probably the pollution. A lot of AI companies are trying to find ways around it, by including watermarks in their model output that their tools can detect, and so on.
We already know that training models on themselves causes model collapse, so OpenAI deciding that they’ve had quite enough wouldn’t be surprising.
On top of that, a lot of websites are clamping down on access because of scraping concerns. Reddit shut down a lot of their third-party APIs, and nicer features, like RSS, Old Reddit (if you haven’t used it in 6 months, you lose access to it, and you need an account for it), etc. Google seems to want a captcha on every search. Twitter famously ramped up the price of its API immensely,.and now has ridiculously stringent measures for accounts, like needing users to download the app, and give hand-scans. It wouldn’t be at all surprising for OpenAI to decide its not worth the effort at this point to try and get around all that, nor pay for it, when they’ve already got all the previous data. Training doesn’t destroy the data used.
A lot of AI training data these days is generated synthetically anyway, and that can be done in-house, whether by their own models, or downloading an open weights model and running it themselves.
the only issue to them is they still need access to current news, like when the ai first came out and you could not convince them that it was current year. it took forever for them to acknowledge charlie kirk’s death. if they are closing the bunker doors their ai is effectively the walking dead
I’m a home-lab newbie and trying to learn Linux basics through self hosting. LLMs have a lot to teach me and I’m already starting to see it’s sloppy errors. One big challenge is that software gets updated and commands change, so not training on new data is going to be back to square 1 for a lot of these use cases.
It was all the goblins