Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] Add support for running tests with stack 9.0.0 #11138

Merged
merged 23 commits into from
Sep 19, 2024

Conversation

mrodm
Copy link
Contributor

@mrodm mrodm commented Sep 16, 2024

Proposed commit message

Add support for running tests with stack 9.0.0.
Add new steps in daily and weekly pipelines for testing packages with stack 9.0.0 (without reporting issues for now).

Author's Checklist

  • Test pipeline with 9.0.0-SNAPSHOT.
  • Add new step to test against stack 9.0.0-SNAPSHOT in daily and weekly pipelines.
  • Ensure daily job for 9.0.0-SNAPSHOT does not report failing tests.
  • Add required configuration for updatecli.
  • Update CI docs with latest changes.

Related issues

@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Sep 16, 2024

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@elasticmachine
Copy link

elasticmachine commented Sep 17, 2024

💔 Build Failed

Failed CI Steps

History

  • 💔 Build #15937 failed c1b56af5106f2b3d44b0fac65b4cf7e30aa2de0a
  • 💔 Build #15936 failed e9e46863d0b67ad73425ded247f879d3c5086eac
  • 💚 Build #15935 succeeded d64f32a0c14651425b4a6dc80d87c95268509a95
  • 💔 Build #15934 failed 0801a94fb685c8f570278c54bf861ce8d8090b18
  • 💚 Build #15933 succeeded 15040d6a5ecbcc6327f42294f33a255205c7cdd9

cc @mrodm

Comment on lines +62 to +64
depends_on:
- step: "check"
allow_failure: false
Copy link
Contributor Author

@mrodm mrodm Sep 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be interesting to run this step after finishing the tests with 8.x ?
It would be updating the step defined in the depends_on field.

To avoid creating another build to test 290 packages at the same time (not sure if that could be an issue). WDYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, this is going to be temporary, I guess that some time after 9.0 is released, 7.x will become unmaintained and we will remove its stage.

If it depends on 8.x, will it be executed if 8.x fails? We want both to be executed even if the other fails, what is pretty common 😅

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it should be able to do that by setting depends_on as:

    depends_on:
      - step: "local-8-stack"
        allow_failure: true

Once it is removed 7.x, it can be reverted the depends_on configuration too.

Copy link
Contributor Author

@mrodm mrodm Sep 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add this for now at least for daily

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But is it a problem to run so many stages at the same time? It looks like we are fixing an issue we don't have (yet?).

I don't have a strong preference with any of the options, but it looks simpler to just run everything at the same time if this is not a problem by now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's true, I'm just moving ahead and probably there would not be any issue.
I'll change it back, and if there are issues we can do this change then.

Comment on lines +40 to +42
depends_on:
- step: "check"
allow_failure: false
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as in the daily job? Would it be interesting to trigger this after the other step testing 8.x has finished? (update depends_on field).

matchpattern: '(STACK_VERSION:) 9\.[^\s]*\.[^\s]*'
replacepattern: '$1 {{ source "latestSnapshot" }}'

update-snapshot-label-weekly:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These steps are now failing in the current check, but they should work once this PR is merged and all the changes are in main branch.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the updatecli config too 👍

@mrodm mrodm marked this pull request as ready for review September 17, 2024 13:56
@mrodm mrodm requested a review from a team as a code owner September 17, 2024 13:56
Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

Comment on lines +62 to +64
depends_on:
- step: "check"
allow_failure: false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, this is going to be temporary, I guess that some time after 9.0 is released, 7.x will become unmaintained and we will remove its stage.

If it depends on 8.x, will it be executed if 8.x fails? We want both to be executed even if the other fails, what is pretty common 😅

if: >
build.source == "trigger_job" && build.env('BUILDKITE_TRIGGERED_FROM_BUILD_PIPELINE_SLUG') == "integrations-schedule-daily"
build.source == "trigger_job" && build.env('BUILDKITE_TRIGGERED_FROM_BUILD_PIPELINE_SLUG') == "integrations-schedule-daily" && build.env('STACK_VERSION') !~ /^9/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

matchpattern: '(STACK_VERSION:) 9\.[^\s]*\.[^\s]*'
replacepattern: '$1 {{ source "latestSnapshot" }}'

update-snapshot-label-weekly:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the updatecli config too 👍

Copy link
Member

@jsoriano jsoriano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! 👍

@elasticmachine
Copy link

elasticmachine commented Sep 18, 2024

⏳ Build in-progress, with failures

Failed CI Steps

History

cc @mrodm

Copy link

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

@mrodm mrodm merged commit 99a22db into elastic:main Sep 19, 2024
4 of 6 checks passed
@mrodm mrodm deleted the update-ci-9.0 branch September 19, 2024 08:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants