subversive_dev

Done: https://github.com/LemmyNet/lemmy/issues/5555

Good evening Dessalines, I have started looking at the posts query.

The lowest hanging fruit I think would be if we could replace some of the joins with WHERE EXISTS which can have a huge impact on the query time. It seems this is supported in Diesel: https://stackoverflow.com/a/74300447

This is my first time looking at the codebase so I can’t tell yet which joins are purely for filtering (in which case they can be replaced by WHERE EXISTS) and which joins need to be left in because some of their columns end up in the final SELECT

I can’t tell for sure yet but it also looks like this might also be using LIMIT...OFFSET pagination? That can be a real drag on performance but isn’t as easy to fix.

EDIT:

Looking some more, and reading some linked github discussion - I think to really get this out of the performance pits will require some denormalization like a materialized view or manual cache tables populated by triggers. I really like the ranking algorithm but so far I’m finding it difficult to optimize from a query perspective

I should have some time tonight to start looking at this. Thanks for the info!

Does the project maintain a list of known slow queries? This is my favorite type of work

Because some accounts like to spam on certain hashtags, I had the best results with muting an account the second I thought it was annoying. It’s nothing personal, I just don’t want to see their posts on my timeline anymore, which is what mute accomplishes

Oh! I see, you are using a mobile browser, that’s what let me astray

Programmers and designers call that sort of pop-up notification a “toast”. I’m guessing whatever app you are using (there isn’t an official Lemmy app right?) is testing out a 3rd-party library for toasts called Toastify. I’m guessing the dev was so excited to see it working that they forgot to remove the test before publishing

You’re welcome!

So if you didn’t care about having to wait for the video to buffer on every scroll, it becomes an easier problem. I kind of think that defeats the purpose of a tiktok-style interface though.

I agree that you wouldn’t necessarily need to build a new algorithm, but like I said, it’s part of the smooth scrolling magic

I’m a dev but not very good at mobile.

I can promise you that a lot of engineering work went into making the tiktok scrolling experience so smooth. Part of the trick is having a good enough algorithm that the user wants to watch the majority of served videos.

Another huge part of it is having lightning fast content distribution and aggressive “prefetching” of the next videos in the feed.

I don’t want to discourage you but I also don’t want you to be caught off guard by the difficulty. Do you want to make this bad enough to give it your nights and weekends for a year?