2023-12-15 - Lemmy Release v0.19.0 - Instance blocking, Scaled sort, and Federation Queue
join-lemmy.org

What is Lemmy?

Lemmy is a self-hosted social link aggregation and discussion platform. It is completely free and open, and not controlled by any company. This means that there is no advertising, tracking, or secret algorithms. Content is organized into communities, so it is easy to subscribe to topics that you are interested in, and ignore others. Voting is used to bring the most interesting items to the top.

Major Changes

This release is very large with almost 400 commits since 0.18.5. As such we can only give a general overview of the major changes in this post, and without going into detail. For more information, read the full changelog and linked issues at the bottom of this post.

Improved Post Ranking

There is a new scaled sort which takes into account the number of active users in a community, and boosts posts from less-active communities to the top. Additionally there is a new controversial sort which brings posts and comments to the top that have similar amounts of upvotes and downvotes. Lemmy’s sorts are detailed here.

Instance Blocks for Users

Users can now block instances. Similar to community blocks, it means that any posts from communities which are hosted on that instance are hidden. However the block doesn’t affect users from the blocked instance, their posts and comments can still be seen normally in other communities.

Two-Factor-Auth Rework

Previously 2FA was enabled in a single step which made it easy to lock yourself out. This is now fixed by using a two-step process, where the secret is generated first, and then 2FA is enabled by entering a valid 2FA token. It also fixes the problem where 2FA can be disabled without passing any 2FA token. As part of this change, 2FA is disabled for all users. This allows users who are locked out to get into their account again.

New Federation Queue

Outgoing federation actions are processed through a new persistent queue. This means that actions don’t get lost if Lemmy is restarted. It is also much more performant, with separate senders for each target instance. This avoids problems when instances are unreachable. Additionally it supports horizontal scaling across different servers. The endpoint /api/v3/federated_instances contains details about federation state of each remote instance.

Remote Follow

Another new feature is support for remote follow. When browsing another instance where you don’t have an account, you can click the subscribe button and enter the domain of your home instance in the popup dialog. It will automatically redirect you to your home instance where it fetches the community and presents a subscribe button. Here is a video showing how it works.

Authentication via Header or Cookie

Previous Lemmy versions used to send authentication tokens as part of the parameters. This was a leftover from websocket, which doesn’t have any separate fields for this purpose. Now that we are using HTTP, authentication can finally be passed via jwt cookie or via header Authorization: Bearer <jwt>. The old authentication method is not supported anymore to simplify maintenance. A major benefit of this change is that Lemmy can now send cache-control headers depending on authentication state. API responses with login have cache-control: private, those without have cache-control: public, max-age=60. This means that responses can be cached in Nginx which reduces server load.

Moderation

Reports are now resolved automatically when the associated post/comment is marked as deleted. This reduces the amount of work for moderators. There is a new log for image uploads which stores uploader. For now it is used to delete all user uploads when an account is purged. Later the list can be used for other purposes and made available through the API.

Cursor based pagination

0.19 adds support for cursor based pagination on the /api/v3/post/list endpoint. This is more efficient for the database. Instead of a query parameter ?page=3, listing responses now include a field "next_page": "Pa46c" which needs to be passed as ?page_cursor=Pa46c. The existing pagination method is still supported for backwards compatibility, but will be removed in the next version.

User data export/import

Users can now export their data (community follows, blocklists, profile settings), and import it again on another instance. This can be used for account migrations and also as a form of backup. The export format is designed to remain unchanged for a long time. You can make regular exports, and if the instance becomes unavailable, register a new account and import the data. This way you can continue using Lemmy seamlessly.

Time zone handling

Lemmy didn’t have any support for timezones, which led to bugs when federating with other platforms. This is now fixed by using UTC timezone for all timestamps.

ARM64 Support

Thanks to help from @raskyld and @kroese, there are now offical Lemmy releases for ARM64 available.

Activity now includes voters

Upgrade instructions

Follow the upgrade instructions for ansible or docker. The upgrade should take less than 30 minutes.

If you need help with the upgrade, you can ask in our support forum or on the Matrix Chat.

Pict-rs 0.5 is also close to releasing. The upgrade takes a while due to a database migration, so read the migration guide to speed it up. Note that Lemmy 0.19 still works perfectly with pict-rs 0.4.

Thanks to everyone

We’d like to thank our many contributors and users of Lemmy for coding, translating, testing, and helping find and fix bugs. We’re glad many people find it useful and enjoyable enough to contribute.

Support development

We (@dessalines and @nutomic) have been working full-time on Lemmy for over three years. This is largely thanks to support from NLnet foundation, as well as donations from individual users.

This month we are running a funding drive with the goal of increasing recurring donations from currently €4.000 to at least €12.000. With this amount @dessalines and @nutomic can each receive a yearly salary of €50.000 which is in line with median developer salaries. It will also allow one additional developer to work fulltime on Lemmy and speed up development.

Read more details in the funding drive announcement.

  • Eskuero@lemmy.fromshado.wsEnglish
    62·
    11 months ago

    Nothing bad would ever happen from deploying on a friday evening right? right?

    YOLO, see you on the other side

  • Ategon@programming.devEnglish
    381·
    11 months ago

    Thanks for merging in some of my changes <3

    Code block support should be great for posts coming from programming.dev and the active user change should help communities feel a bit more alive and be more accurate to actual activity

  • chaorace@lemmy.sdf.orgEnglish
    181·
    11 months ago

    It’s always a treat to see the wheels of open source & community funding meshing together to bring about a better internet. You and @[email protected] have already done the honors of thanking the open source contributors, so I think it’s now my turn as an ordinary user to thank you both for your continued hard work and leadership – I have my quibbles, but these do not preclude the giving of a well-deserved and hearty thanks: Thank you! It continues to please me greatly to have the privilege (in both senses of the word) to contribute to the financial sustainability of the project as one of its many small recurring donators.

    And, while I’m at it… since the opportunity doesn’t come up often: I also want to specifically express gratitude to dessalines for their (unrelated) work on the thumb-key project. As a long-time user of MessagEase (10+ years!), I’ve become a happy convert to your more well-maintained and open source replacement. So, thanks for that too! Please continue to suffer my past & future criticisms with the knowledge that it always originates from a place of equal parts respect & gratitude.

    • Dessalines@lemmy.mlOPMEnglish
      2·
      11 months ago

      No probs, and thx for your support! I’m glad when anyone finds the things I work on useful.

      We still have far to go privacy-wise, but finally there’s FOSS alternatives for nearly every major app and comms platform. IMO the next big one that still needs to be tackled is youtube, and lemmy could have a part to play there.

      • v_krishna@lemmy.mlEnglish
        1·
        11 months ago

        Re thumb-key do you have recommended tutorials for getting comfortable with it? I found trying to do touch typing tutorials didn’t really help, both because they are generally made for desktop environments and they are geared towards qwerty layout (e.g., get comfortable with home row first etc). I tried forcing myself to use it for a full 24 hours as the concept makes a ton of sense to me, but got very frustrated with myself and then dug into the world of which layout to choose, got overwhelmed, and switched back to whatever this qwerty layout that samsung one ui provides on galaxys.

        • Dessalines@lemmy.mlOPMEnglish
          2·
          11 months ago

          Just what’s on the readme guide. If you’re in english, I’d just recommend using the default layout and settings for ~3 days, that seems to be the time to get used to it. Or practice on monkeytype.com

      • possibly a cat@lemmy.mlEnglish
        0·
        11 months ago

        I love the webtor idea, especially embedded.

        I’m sure the UX would experience growing pains but most of that would be for the creators, who are more likely to be power users and willing to learn a new tool. With so little risk to the core project (i.e., eating through server costs) you wouldn’t need mass/rapid adoption to make it a sustainable feature.

        Have you considered surveying stakeholders (primarily the userbase) about features and strategy? I know that can be a double-edged blade. But I’m curious what %s would prefer development toward existing and in-progress feature sets, vs. expanding the feature scope to cover more of the mediascape (e.g., webtor-based YT replacement).

        Awesome work to you and nutonic, as well as the rest of the dev team and other contributors. I’m very grateful.

        • Dessalines@lemmy.mlOPMEnglish
          1·
          11 months ago

          Thx! Ya its sad that torrents have effectively solved the static data distribution problem, but we still don’t have user-friendly ways to access them via browsers. We’ve got tons of pictures, video, music, and other large files still being mainly hosted on centralized platforms like youtube and spotify, fully under their control. Its a sad state of affairs.

          Have you considered surveying stakeholders (primarily the userbase) about features and strategy?

          We sometimes use the github Sort by thumbs up to find the most-wanted issues, and bug fixing is always necessary… but part of being open-source means that it should be fun, and its nice to be able to pick issues we feel motivated to work on. If we didn’t have that, we’d get burnt out pretty fast.

          • possibly a cat@lemmy.mlEnglish
            1·
            11 months ago

            but part of being open-source means that it should be fun, and its nice to be able to pick issues we feel motivated to work on. If we didn’t have that, we’d get burnt out pretty fast.

            I 100% understand; that’s part of what I meant by double-edged sword.

            I’m happy to be a user wherever that passion guides you!

  • SorteKanin@feddit.dkEnglish
    161·
    11 months ago

    These are great features, thanks for all the work and congratulations on the release.

    I will say though - in general, it would probably be easier and safer to do more incremental releases. It has been a long time since 0.18.5.

    Have you considered adopting a release train model, similar to what Rust does? The Bevy game engine project has adopted such a model as well and it has worked well as far as I gather. The idea is basically to release on a fixed time interval, forever. No more surprises about when things release and there’s a convenient regular opportunity to encourage people to donate :)

    • interolivary@beehaw.orgEnglish
      4·
      11 months ago

      To expand on this, the problem with huge releases is that bugs tend to pile up along with the new features, simply because more stuff has been changed. You’re much more likely to experience a painful release if you pile in months of changes, and this is doubly true for a distributed system with no rigorous testing of the system itself instead of just units

      • SorteKanin@feddit.dkEnglish
        31·
        11 months ago

        Yes, 100%. That’s what I meant by “safer” but thanks for expanding :)

    • Dessalines@lemmy.mlOPMEnglish
      2·
      11 months ago

      Thx! I’d also like to do more frequent releases, and would especially like a monthly / regular release schedule. It gets difficult for network services like this one, where we have a lot of apps speaking the same language, and then we need to change or remove a bunch of words (IE breaking changes). So this one was mainly pushed back due to the large number of breaking changes, with us wanting to get them all into one release. Otherwise devs would be tearing their hair out every other week accounting for yet another breaking change.

      Once the API stabilizes and reaches a 1.0 status, breaking changes should be far less frequent, and more frequent releases should be doable.

  • cheer@lemmy.worldEnglish
    161·
    11 months ago

    Everything added is soo helpful. Congrats to the team and everyone else who contributed.

  • sabreW4K3@lemmy.tfEnglish
    101·
    11 months ago

    Congratulations on finally getting this out. The amount of work that went into this release was amazing. Thank you everyone that contributed.

  • kratoz29@lemm.eeEnglish
    8·
    11 months ago

    Excellent! Hopefully it doesn’t break 3rd party apps 🙃

    Good thing I have plenty huh?

    • stebo02@sopuli.xyzEnglish
      1·
      11 months ago

      It doesn’t break boost but you have to log out and log in again to be able to comment or post