cross-posted from: https://lemmy.management/post/665809

I made this tool to help self-hosters, new admins, or smaller instances have more global and updated content on their instances.

This is the similar to Lemmy Community Seeder but is designed to be run periodically to capture new communities, and include EVERYTHING by default.

  • PriorProject@lemmy.world
    link
    fedilink
    English
    arrow-up
    0
    ·
    edit-2
    1 year ago

    This is a terrible idea, and borderline irresponsible. One of the key reasons that Lemmy doesn’t subscribe by default is to avoid forcing servers with many communities to waste time/CPU delivering messages to servers where no one will read those messages. By subscribing to everything, you’re telling all those overloaded servers to waste time sending content to your server that you’ll never even see.

    • It also will massively inflate your db by multiple GB/day.
    • It will maximize the chances of you downloading and hosting copyright infringing content and content that may be illegal in your jurisdiction but not in the jurisdiction where it’s hosted (loli, etc).

    It is much MUCH better to just hit lemmyverse.net and subscribe to 10-100 communities you care about. If script accepted a list of community-urls and automated subscribing to those, that would be super nice. Subscribing to the entire lemmyverse is terrible for your server, for your hosting liability, and for the lemmyverse’s performance.

    • calvin@lemmy.todayyoutomorrow.me
      cake
      link
      fedilink
      English
      arrow-up
      0
      ·
      1 year ago

      The idea of this app, which is desperately needed, is to allow those of us that are running a very small private instance to still get access to all of the communities. If you have a decent user base then you don’t need this anymore as the users themselves will provide the functionality.

      So in my case I am the only user on my instance so I am certainly not going to be hammering a bunch of instances just to send me updates of whatever total number of communities I’m subscribed to.

      • PriorProject@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 year ago

        So in my case I am the only user on my instance so I am certainly not going to be hammering a bunch of instances just to send me updates of whatever total number of communities I’m subscribed to.

        You don’t understand how federated replication works. It doesn’t occur on-demand when you read a post, it occurs when the instance hosting the community gets a post, comment, or vote. The federation load you place on other servers has nothing to do with how many users are on your instance or how much they read… it has everything to do with how many communities they subscribe to. This script is literally signing you up to proactively receive the firehose of every post and comment in the lemmyverse, without regard for what you actually look at.

        I completely understand the idea of the app, and your confusion about how much load it generates is exactly why it’s such an irresponsible idea. If you want to fill the timeline of your small instance, do so by subscribing to specific communities you’re interested in until your timeline becomes active enough for you. Subscribing to 100 communities you care about will result in a very lively feed of stuff that is interesting to you, while generating a tiny percentage of the federation load this approach does. Carpet bombing the entire lemmyverse with subscriptions you cannot read is madness. It’s like writing a reddit app that downloads everything ever posted to reddit to your phone to save you the trouble of picking subreddits to follow. It’s bad for reddit, bad for your phone, bad for your isp, and a bad idea all around. If I were running a large instance, I’d defederate with any tiny instance I observed subscribing indiscriminately via this script. It’s abuse.