Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Improve test providers coverage and prepare for 98.0 release #3458

Merged
merged 5 commits into from
Jul 14, 2022
Merged

CI: Improve test providers coverage and prepare for 98.0 release #3458

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
294ca16
CI: pin the versions of tp-qemu and tp-libvirt
clebergnu Jul 14, 2022
7cd09e0
CI: increase integration with test providers coverage
clebergnu Jul 14, 2022
d5741fa
CI: include the path of avocado-runner-avocado-vt on list
clebergnu Jul 14, 2022
ea320e1
CI: increase testing coverage of "avocado devel" tasks
clebergnu Jul 14, 2022
53e8f1e
CI: remove the waiver of failures with latest Avocado
clebergnu Jul 14, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions .cirrus.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,19 +28,24 @@ fedora_35_task:
- test -z $AVOCADO_SRC || python3 -m pip install $AVOCADO_SRC
- python3 $SETUP
bootstrap_script: &bootstrap_scr
# Pin the versions of tp-qemu and tp-libvirt to known working versions.
# These versions can be bumped at any time, but provide better understanding
# of regressions or version update requirements on the test providers side.
- python3 -c "from virttest.asset import ConfigLoader; cl = ConfigLoader('virttest/test-providers.d/io-github-autotest-qemu.ini'); cl.set('provider', 'ref', '33ed9b0ac6e4255877ff109a1d91fa84c9097dc5'); cl.save()"
- python3 -c "from virttest.asset import ConfigLoader; cl = ConfigLoader('virttest/test-providers.d/io-github-autotest-libvirt.ini'); cl.set('provider', 'ref', 'e6af7700f879fe1547bfb1421439121d35c919bd'); cl.save()"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with you that it is important to pin the versions of tp-qemu and tp-libvirt, so we will be changing only one component at the time. But IMO we still need to update these version once in a while, and I am missing some well-defined rules how this will be done. Something like release check that, the tp-qemu and tp-libvirt were updated to the newest version in every release.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The exact release steps need to be better ironed out indeed on the Avocado-VT side. I can follow this up with a documentation change proposal, or open a discussion about it.

But I think both would take a lot more time (because of the discussion) than what is needed for a release related change.

Let me know if you are thinking of something simpler that can be addressed here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, I agree with you that this needs to be well discuss, so I am ok with doing this after the release. Can you please open an issue or discussion about it, so we don't forget about this? Thanks

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. The issue tracking this is #3462 .

- python3 -m avocado vt-bootstrap --vt-type=$VT_TYPE --vt-skip-verify-download-assets --yes-to-all
list_script: &list_scr
- python3 -m avocado list --vt-save-config=/tmp/config --vt-type=$VT_TYPE -- boot | tee /tmp/list
- PATH=~/.local/bin:$PATH python3 -m avocado list --vt-save-config=/tmp/config --vt-type=$VT_TYPE -- boot | tee /tmp/list
- test $VT_TYPE == "qemu" || test $(wc -l < /tmp/list) -gt 50
- test $VT_TYPE == "libvirt" || test $(wc -l < /tmp/list) -eq 1
- python3 -m avocado list --vt-config=/tmp/config --vt-type=$VT_TYPE -- boot | tee /tmp/list_from_config
- PATH=~/.local/bin:$PATH python3 -m avocado list --vt-config=/tmp/config --vt-type=$VT_TYPE -- boot | tee /tmp/list_from_config
- diff /tmp/list /tmp/list_from_config
- python3 -m avocado vt-list-guests --vt-type=$VT_TYPE
- python3 -m avocado vt-list-archs --vt-type=$VT_TYPE
dry_run_script: &dry_run_scr
- PATH=~/.local/bin:$PATH python3 -m avocado --show all run --vt-type=$VT_TYPE --dry-run -- io-github-autotest-qemu.boot
run_boot_script: &run_boot_scr
- test $VT_TYPE == "libvirt" || PATH=$HOME/.local/bin:$PATH python3 -m avocado --show all run --vt-type=$VT_TYPE --vt-extra-params nettype=user -- io-github-autotest-qemu.boot
run_tests_script: &run_tests_scr
- test $VT_TYPE == "libvirt" || PATH=$HOME/.local/bin:$PATH python3 -m avocado --show all run --vt-type=$VT_TYPE --vt-extra-params nettype=user -- io-github-autotest-qemu.boot io-github-autotest-qemu.qemu_disk_img.convert.base_to_raw io-github-autotest-qemu.commit_with_backing.default

centos_8_1_task:
container:
Expand All @@ -67,7 +72,6 @@ avocado_devel_task:
container:
image: quay.io/avocado-framework/avocado-vt-ci-fedora-35
kvm: true
allow_failures: $AVOCADO_SRC == 'git+https://github.com/avocado-framework/avocado#egg=avocado_framework'
env:
matrix:
# Latest Avocado
Expand All @@ -93,3 +97,5 @@ avocado_devel_task:
*list_scr
dry_run_script:
*dry_run_scr
run_tests_script:
*run_tests_scr