fix: extremely slow paginated conversation list query [WPB-11808] 🍒 #3102
Conversation
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
github-actions
bot
added
cherry-pick
|
|
Bencher Report| Branch | fix/too-slow-paginated-conversation-list-query-cherry-pick |
| Testbed | ubuntu-latest |
⚠️ WARNING: The following Measure does not have a Threshold. Without a Threshold, no Alerts will ever be generated!Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the--ci-only-thresholdsCLI flag.
Click to view all benchmark results
| Benchmark | Latency | nanoseconds (ns) |
|---|---|---|
| com.wire.kalium.benchmarks.logic.CoreLogicBenchmark.createObjectInFiles | 📈 view plot | 667,688.90 |
| com.wire.kalium.benchmarks.logic.CoreLogicBenchmark.createObjectInMemory | 📈 view plot | 355,288,301.45 |
| com.wire.kalium.benchmarks.persistence.MessagesNoPragmaTuneBenchmark.messageInsertionBenchmark | 📈 view plot | 1,304,093,985.81 |
| com.wire.kalium.benchmarks.persistence.MessagesNoPragmaTuneBenchmark.queryMessagesBenchmark | 📈 view plot | 21,978,820.06 |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## develop #3102 +/- ##
===========================================
+ Coverage 54.19% 54.21% +0.02%
===========================================
Files 1196 1196
Lines 35592 35596 +4
Branches 3625 3625
===========================================
+ Hits 19288 19298 +10
+ Misses 14900 14899 -1
+ Partials 1404 1399 -5
... and 2 files with indirect coverage changes Continue to review full report in Codecov by Sentry.
|
Datadog ReportBranch report: ✅ 0 Failed, 3164 Passed, 107 Skipped, 37.09s Total Time |
saleniuk
requested review from
a team,
typfel,
yamilmedina,
alexandreferris,
saleniuk and
damian-kaczmarek
and removed request for
a team
saleniuk
deleted the
fix/too-slow-paginated-conversation-list-query-cherry-pick
branch
MohamadJaara
added
echoes: technical-roadmap
This PR was automatically cherry-picked based on the following PR:
Original PR description:
PR Submission Checklist for internal contributors
The PR Title
SQPIT-764The PR Description
What's new in this PR?
Issues
The paginated conversation list query is unable to show the list in a reasonable time, if someone has many conversations and messages then it can take multiple minutes.
Causes (Optional)
Left joining messages with multiple columns from different content tables and left joining unread events to just get last message for each conversation and count of different unread events is too far from optimal. We have managed to reduce full table scans but there is still a lot of data to work on by this query.
Solutions
To have a proper data to work on, I filled a database with ~1150 conversations and over 200 000 messages, biggest conversation had around 35 000 messages and 27 000 unread events.
With this DB, I waited more than 10 minutes using the current paginated query to get the result and eventually gave up. So I checked various ideas, below are the ones that looked most promising.
For unread events table, there are two ways: to left join all unread events into main query and calculate it or to make a subquery or view to first count events on just that table and then left join results of that calculations into the main query.
For message table, first idea was to match the solution implemented some time ago and to make a single subquery to get ids of last messages for each conversation and then left join it with all contents and other things - thanks to that it doesn't left join all messages and their contents first, but just the messages that it actually needs.
Second one was to create a dedicated table which contains ids of last message for each conversation, and just that, so it could replace this subquery and make the query even simpler. The main problem with that was to make sure this table is synced with the message table, so 3 triggers were created - when inserting new message, when updating visibility or type of a message (two parameters that are taken into account when selecting last message) and when deleting a last message (to find the one that from now on should be the last one). Another issue is whether this can affect the event handling times, so I checked that by executing 3000x
processApplicationMessageusing current code and with this additional table which needs to stay synced using triggers when new message is inserted:Results show that there is no difference, so it's safe to use.
To have a perspective, here are the times of three main queries used to fetch conversation list currently without the pagination:
Around 175ms to get the list without pagination.
As I said, with current paginated query I couldn't get a result in over 10 minutes, but with improvements explained above, it started to work in a very reasonable time:
A, B, C, D are just a combinations of these improvements for joining both message and unread event table, explained here (with average execution times):
It looks like the winner is D, and it's unbelievable that the execution of such complex query can take under 20ms, but so far it looks like everything is working correctly.
So here in this PR, the D approach is implemented - with new dedicated table
LastMessageandUnreadEventCountsGroupedview used in the main query which doesn't require anymore to be GROUPed BY.Testing
How to Test
Enable
paginated_conversation_list_enabledfeature flag and open the conversation list.PR Post Submission Checklist for internal contributors (Optional)
PR Post Merge Checklist for internal contributors
References
feat(conversation-list): Sort conversations by most emojis in the title #SQPIT-764.