RoundSparrow

This basically shuts my idea down

it’s not very difficult to modify the code for something like this… and closing off registration wont’ let anyone else login and create new content form your istance.

Personally the load on the major servers by having one more instance that subscribes to everything is why I think people should back off from creating more than the 1500 instances Lemmy network already has. Delivery of every single vote, comment, post 24 hours a day just so one person can read content for an hour or two a day.

That makes sense for email systems where all that content doesn’t have to be sent, but for Lemmy it’s a huge amount of overhead.

Keep in mind that you’re going to be retrieving and storing a huge amount of data running these scripts

And you are adding to the overload of lemmy.world, beehaw, lemmy.ml, etc who have all the popular content communities. Federation has a lot of overhead, as does having to distribute a community one vote at a time to 500 subscribed servers.

pend my time on Lemmy scrolling “All”, which I think is a pretty common thing.

There was a lot of advice handed out back in June that the answer to scaling Lemmy was to go create instances. The reason it works is because “All” is empty on a virgin system ;) With no data in the database, the logic Lemmy hands to PostgreSQL works real fast ;)

I use that periodically to compare feeds, and like I said sometimes a post or comment is missing, actually I often see a comment that looks like it’s responding to a another comment, but I cannot see the parent comment.

There have been bugs in Lemmy not sending comment deletes to all the instances. And lemmy.world and lemmy.ml were not communicating fro Saturday through Tuesday. Lemmy.world had some significant outages. It gets pretty tricky to track down and identify exact causes while things are unstable.

This year has been dramatic. I’ve seen a big increase of users with quality content doing deletes in protest of Reddit. And the shift to sites like Lemmy that are not as favored by search engines.

Reddit should have gone the other direction, become a non-profit, eliminate advertising, go back to open sourcing the code like they used to, and run on donations. Cut their staff of people that had anything to do with advertising and trying to market the platform.

For open source, I almost always found IRC was a black hole of information. All kinds of developers discussing things that never made it to search engines. It’s a long tradition.

Matrix chat as the core developer hangout I find is similar

FYI, some major PostgreSQL optimization has been found with the stored procedures: https://github.com/LemmyNet/lemmy/pull/3704

This should really help with Beehaw and other sites that have local users creating posts and comments, as the runaway site_aggregates queries were creating unnecessary writes.

Lemmy is written in Rust, I don’t think any other ActivePub platform uses Rust.

Lemmy is written in Rust, I don’t think any other platform uses Rust.

Up until early July, Lemmy was damned if you do, damned if you don’t. Federation had massive performance overhead due to some bugs and each additional instance that went online and subscribed to the big 4 popular servers was causing an even worse load problem than if say 30 users had joined directly. Especially instances that wanted a fully populated All listing, that meant every single thing was being sent to the server even if nobody was really reading that stuff.

And things like searching for topic content are going to be pretty limited given these newer servers don’t have much history.

The aftermath of this attempt to scale is that there is also likely a lot of duplicate data, conversations that are mostly repetitive and posts to the same topics. Let alone the bugs Lemmy has federating deletes and moderation removal that doesn’t impact direct users on the main servers as much.

posts I create on lemmy.secondpartysoftware.com don’t show up on the others.

your server isn’t going to send copies to other servers unless an account on that remote server subscribes. You said you created a new community, nobody is going to be subscribed.

your post got duplicated, this is the 2nd copy

open APIs for other instances to be able to access the content of your instance in order to make federation possible.

the federation API is independent of the front-end client API. You can run headless, without lemmy-ui, and federation still works. The API structure for federation is standardized, the front-end client API is unique to Lemmy.

I’m finding issues with communities not loading posts, comments or general sync issues.

Lemmy’s Rust code uses an ORM called Diesel that masks the SQL statements and you really have to watch the PostgreSQL server independently to verify that the SQL isn’t doing wild things like loading thousands of records when you only needed 3. Just today people are finally sharing some information out of the big servers (lemmy.world) as to what PostgreSQL side says is actually happening. Hopefully the biggest mistakes are going to get cleaned up quickly.