Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bugfixes, logging improvements for LLO (and update chainlink-data-streams) #15169

Merged
merged 5 commits into from
Nov 19, 2024
Merged

Bugfixes, logging improvements for LLO (and update chainlink-data-streams) #15169

merged 5 commits into from
Nov 19, 2024

Conversation

samsondav
Copy link
Collaborator

Requires

Supports

@samsondav samsondav requested review from a team as code owners November 8, 2024 15:33
@samsondav samsondav requested a review from Atrax1 November 8, 2024 15:33
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 8, 2024 •
edited
Loading

AER Report: CI Core ran successfully ✅

aer_workflow , commit

AER Report: Operator UI CI ran successfully ✅

aer_workflow , commit

@samsondav samsondav requested review from a team as code owners November 11, 2024 17:29
@samsondav samsondav changed the title Some logging improvements for LLO Some logging improvements for LLO (and update chainlink-data-streams) Nov 11, 2024
@samsondav samsondav added release-preview build-publish Build and Publish image to SDLC and removed release-preview labels Nov 11, 2024
@samsondav samsondav enabled auto-merge November 12, 2024 13:38
@samsondav samsondav force-pushed the llo_fixes branch 3 times, most recently from 8546884 to 216b675 Compare November 12, 2024 20:30
martin-cll
martin-cll previously approved these changes Nov 13, 2024
View reviewed changes
@samsondav samsondav force-pushed the llo_fixes branch 3 times, most recently from e02b354 to 301c1a2 Compare November 14, 2024 14:53
@github-actions GitHub Actions
Copy link
Contributor

Flaky Test Detector for ./go.mod project has failed ❌

Ran new or updated tests between develop and e02b354 (llo_fixes).

View Flaky Detector Details | Compare Changes

Failed Tests

Ran 191 tests in total for all affected test packages. Below are the tests identified as flaky, with a pass ratio lower than the 100% threshold:

TestPackage                                                       TestName                               PassRatio  RunCount   Skipped
---------                                                         ---------                              ---------  ---------  ---------
github.com/smartcontractkit/chainlink/v2/core/services/ocrcommon  TestCollectMercuryEnhancedTelemetryV2  0%         1          false

@samsondav samsondav requested a review from a team as a code owner November 14, 2024 16:13
@samsondav
Copy link
Collaborator Author

This PR doesn't actually fix flakey tests but it does make them easier to identify

brunotm
brunotm previously approved these changes Nov 19, 2024
View reviewed changes
core/services/llo/telemetry.go
// feed) so we need to make sure the buffer is large enough.
//
// 2000 feeds * 5s/250ms = 40_000 should hold ~5s of buffer in the worst case.
chTelemetryObservation := make(chan TelemetryObservation, 40_000)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we make the buffer length increase with the concurrency setting? (eg TransmitConcurrency * 200)?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think transmit concurrency affects it directly. We might consider making it configurable but I'd honestly prefer to choose a large enough conservative value that we don't have to worry about it, the only real penalty is using some extra memory

@github-actions GitHub Actions
Copy link
Contributor

Flaky Test Detector for github.com/smartcontractkit/chainlink project has failed ❌

Ran new or updated tests between develop and 2636441 (llo_fixes).

View Flaky Detector Details | Compare Changes

Flaky Tests

Ran 320 unique tests. Below are the tests identified as flaky, with a pass ratio lower than the 99% threshold:

TestPackage                                                              TestName                                                                                                                                                                                      PassRatio  RunCount   Skipped
---------                                                                ---------                                                                                                                                                                                     ---------  ---------  ---------
github.com/smartcontractkit/chainlink/v2/core/services/ocr2/plugins/llo  TestIntegration_LLO                                                                                                                                                                           67%        3          false
github.com/smartcontractkit/chainlink/v2/core/services/ocr2/plugins/llo  TestIntegration_LLO/using_legacy_verifier_configuration_contract,_produces_reports_in_v0.3_format/receives_at_least_one_report_per_channel_from_each_oracle_when_EAs_are_at_100%_reliability  67%        3          false
github.com/smartcontractkit/chainlink/v2/core/services/ocr2/plugins/llo  TestIntegration_LLO/Blue/Green_lifecycle_(using_JSON_report_format)                                                                                                                           67%        3          false
github.com/smartcontractkit/chainlink/v2/core/services/ocr2/plugins/llo  TestIntegration_LLO/using_legacy_verifier_configuration_contract,_produces_reports_in_v0.3_format                                                                                             67%        3          false

@samsondav samsondav requested a review from a team as a code owner November 19, 2024 16:04
@samsondav samsondav requested a review from brunotm November 19, 2024 17:09
@samsondav samsondav added this pull request to the merge queue Nov 19, 2024
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 19, 2024
@samsondav samsondav added this pull request to the merge queue Nov 19, 2024
Merged via the queue into develop with commit 1341a3a Nov 19, 2024
149 checks passed
@samsondav samsondav deleted the llo_fixes branch November 19, 2024 19:50
This was referenced Nov 19, 2024
Draft
Draft
cedric-cordenier pushed a commit that referenced this pull request Nov 20, 2024
@samsondav @cedric-cordenier
Bugfixes, logging improvements for LLO (and update chainlink-data-str…
c588102 
…eams) (#15169)

* Logging and misc LLO fixes

Fixes: MERC-6626

* Quieten down enhanced EA telemetry logging

* Quieten workflow pruner logging

* Fix test

* Fix lint
@github-actions github-actions bot mentioned this pull request Nov 26, 2024
Draft
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@denis-chernov-smartcontract denis-chernov-smartcontract denis-chernov-smartcontract approved these changes

@bolekk bolekk bolekk approved these changes

@jmank88 jmank88 jmank88 approved these changes

@Atrax1 Atrax1 Awaiting requested review from Atrax1 Atrax1 is a code owner automatically assigned from smartcontractkit/foundations

@msuchacz-cll msuchacz-cll Awaiting requested review from msuchacz-cll

@martin-cll martin-cll Awaiting requested review from martin-cll

@brunotm brunotm Awaiting requested review from brunotm

Assignees
No one assigned
Labels
build-publish Build and Publish image to SDLC
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@samsondav @jmank88 @bolekk @brunotm @martin-cll @denis-chernov-smartcontract @msuchacz-cll