gayhitler420@lemm.eetoTechnology@beehaw.org•The rise and fall of robots.txt: As unscrupulous AI companies seek out more and more data, the basic social contract of the web is falling apart.
61·
9 months agorobots.txt isn’t a basic social contract, it’s a file intended to save web crawlers precious resources.
Of course they want the model collapse. Literally no American tech company has been about reliably, sustainably supplying a good or service or stewarding some public good.
They’re doing the vc -> juice stock -> gut resources cycle. Nobody cares about the model.