![](/static/253f0d9b/assets/icons/icon-96x96.png)
![](https://lemmy.world/pictrs/image/0d5e3a0e-e79d-4062-a7bc-ccc1e7baacf1.png)
I don’t think (completely wild guess here) AI content crawlers should have any more impact than the dozens and dozens of search spiders that make up must of my own site’s traffic.
The impact was magnified for Twitter because it generates so much new content every second. That wasn’t an issue when Twitter had a nice, properly cached API and it shouldn’t be an issue for fediverse instances going forward because we have RSS and caching and we’re not so stupid as to turn those off. Like, what kind of moron would do that?
It’s a great predictor of how the rest of your interaction with a site will go. If a dev doesn’t have the space and motivation to give a shit about details like this, they’ll cut corners on other shit too. It doesn’t always mean they’re bad developers; more often they’re just rushed.