Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add metrics for sync engine slow and incremental (WPB-11199) #3086

Merged
merged 9 commits into from
Nov 5, 2024

Conversation

yamilmedina
Copy link
Contributor

@yamilmedina yamilmedina commented Nov 1, 2024

StoryWPB-11199 [Android] add Datadog state for sync job start and stop and how much time it took


PR Submission Checklist for internal contributors

  • The PR Title

    • conforms to the style of semantic commits messages¹ supported in Wire's Github Workflow²
    • contains a reference JIRA issue number like SQPIT-764
    • answers the question: If merged, this PR will: ... ³
  • The PR Description

    • is free of optional paragraphs and you have filled the relevant parts to the best of your ability

What's new in this PR?

Issues

We are somehow blind in some places regarding sync

Causes (Optional)

Difficulty to troubleshoot sync problems, and no metrics regarding this, like we have in other places for example for events.

Solutions

  • Create a wrapper class for handling formatted logs for sync manager
  • Adds metrics handling data, like id and duration of sync jobs
  • Call from Slow and Incremental sync to these logs
  • Incremental sync needs to handle foreground case and background cases, that's why the state change handles the ids and started time of job across coroutine collections using .runningFold

Testing

Test Coverage (Optional)

  • I have added automated test to this contribution

How to Test

Interact with the app and see logs formatted in Datadog (not available if you don't have local setup of credentials)

Notes (Optional)

Dashboards in DD will follow.
https://app.datadoghq.eu/dashboard/crc-suh-fq6/android-sync-manager-performance?fromUser=false&refresh_mode=sliding&from_ts=1730463823807&to_ts=1730478223807&live=true


PR Post Submission Checklist for internal contributors (Optional)

  • Wire's Github Workflow has automatically linked the PR to a JIRA issue

PR Post Merge Checklist for internal contributors

  • If any soft of configuration variable was introduced by this PR, it has been added to the relevant documents and the CI jobs have been updated.

References
  1. https://sparkbox.com/foundry/semantic_commit_messages
  2. https://github.com/wireapp/.github#usage
  3. E.g. feat(conversation-list): Sort conversations by most emojis in the title #SQPIT-764.

Copy link
Contributor

github-actions bot commented Nov 1, 2024

Test Results

3 255 tests  ±0   3 149 ✅ ±0   4m 35s ⏱️ +26s
  556 suites ±0     106 💤 ±0 
  556 files   ±0       0 ❌ ±0 

Results for commit 4de67e7. ± Comparison against base commit 1f3c946.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Nov 1, 2024

🐰 Bencher Report

Branchfeat/add-metrics-sync-engine
Testbedubuntu-latest

⚠️ WARNING: The following Measure does not have a Threshold. Without a Threshold, no Alerts will ever be generated!

Click here to create a new Threshold
For more information, see the Threshold documentation.
To only post results if a Threshold exists, set the --ci-only-thresholds CLI flag.

Click to view all benchmark results
BenchmarkLatencynanoseconds (ns)
com.wire.kalium.benchmarks.logic.CoreLogicBenchmark.createObjectInFiles📈 view plot
⚠️ NO THRESHOLD
676,111.15
com.wire.kalium.benchmarks.logic.CoreLogicBenchmark.createObjectInMemory📈 view plot
⚠️ NO THRESHOLD
604,272,249.91
com.wire.kalium.benchmarks.persistence.MessagesNoPragmaTuneBenchmark.messageInsertionBenchmark📈 view plot
⚠️ NO THRESHOLD
944,137,325.81
com.wire.kalium.benchmarks.persistence.MessagesNoPragmaTuneBenchmark.queryMessagesBenchmark📈 view plot
⚠️ NO THRESHOLD
21,522,041.55
🐰 View full continuous benchmarking report in Bencher

@codecov-commenter
Copy link

codecov-commenter commented Nov 1, 2024

Codecov Report

Attention: Patch coverage is 95.45455% with 2 lines in your changes missing coverage. Please review.

Project coverage is 52.73%. Comparing base (1f3c946) to head (4de67e7).

Files with missing lines Patch % Lines
...in/com/wire/kalium/logic/sync/SyncManagerLogger.kt 93.93% 2 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3086      +/-   ##
===========================================
+ Coverage    52.69%   52.73%   +0.03%     
===========================================
  Files         1320     1321       +1     
  Lines        51166    51206      +40     
  Branches      4746     4747       +1     
===========================================
+ Hits         26963    27001      +38     
- Misses       22276    22279       +3     
+ Partials      1927     1926       -1     
Files with missing lines Coverage Δ
...m/logic/sync/incremental/IncrementalSyncManager.kt 86.73% <100.00%> (+0.56%) ⬆️
...com/wire/kalium/logic/sync/slow/SlowSyncManager.kt 93.02% <100.00%> (+0.25%) ⬆️
...in/com/wire/kalium/logic/sync/SyncManagerLogger.kt 93.93% <93.93%> (ø)

... and 2 files with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1f3c946...4de67e7. Read the comment docs.

@datadog-wireapp
Copy link

datadog-wireapp bot commented Nov 1, 2024

Datadog Report

Branch report: feat/add-metrics-sync-engine
Commit report: 9f44582
Test service: kalium-jvm

✅ 0 Failed, 3149 Passed, 106 Skipped, 33.86s Total Time

Copy link

sonarqubecloud bot commented Nov 4, 2024

@yamilmedina yamilmedina enabled auto-merge November 5, 2024 12:51
@yamilmedina yamilmedina added this pull request to the merge queue Nov 5, 2024
Merged via the queue into develop with commit 4667ea7 Nov 5, 2024
22 checks passed
@yamilmedina yamilmedina deleted the feat/add-metrics-sync-engine branch November 5, 2024 17:04
yamilmedina added a commit that referenced this pull request Nov 5, 2024
…3086)

* feat: add base for metrics, implement in slow sync

* feat: add base for metrics, implement in slow sync

* feat: add metrics for slow sync and improve json log

* feat: incremental sync logs

* feat: incremental sync logs

* feat: incremental sync logs, improv for foreground

* feat: refactor some naming to clarify

* feat: refactor some naming to clarify

* feat: docs
@echoes-hq echoes-hq bot added the echoes: product-roadmap Work aligned with the customer-announced roadmap, targeting a specific release date. label Nov 6, 2024
github-merge-queue bot pushed a commit that referenced this pull request Nov 7, 2024
…3086) (#3090)

* feat: add base for metrics, implement in slow sync

* feat: add base for metrics, implement in slow sync

* feat: add metrics for slow sync and improve json log

* feat: incremental sync logs

* feat: incremental sync logs

* feat: incremental sync logs, improv for foreground

* feat: refactor some naming to clarify

* feat: refactor some naming to clarify

* feat: docs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
echoes: product-roadmap Work aligned with the customer-announced roadmap, targeting a specific release date. 👕 size: M type: feature ✨
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants