ksm_ksmtuned: gives more time for ksm to start #3957

mcasquer · 2024-02-08T10:21:18Z

ksm_ksmtuned: gives more time for ksm to start

After checking the timings, the case failed because
ksmtuned needs some more time in order to start ksm
after reaching the threshold.

Signed-off-by: mcasquer [email protected]
ID: 1932

mcasquer · 2024-02-08T10:23:20Z

Checked 5 times, the current increase should be enough

 (1/5) repeat1.Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.ksm_ksmtuned.q35: STARTED
 (1/5) repeat1.Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.ksm_ksmtuned.q35: PASS (344.12 s)
 (2/5) repeat2.Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.ksm_ksmtuned.q35: STARTED
 (2/5) repeat2.Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.ksm_ksmtuned.q35: PASS (297.25 s)
 (3/5) repeat3.Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.ksm_ksmtuned.q35: STARTED
 (3/5) repeat3.Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.ksm_ksmtuned.q35: PASS (340.82 s)
 (4/5) repeat4.Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.ksm_ksmtuned.q35: STARTED
 (4/5) repeat4.Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.ksm_ksmtuned.q35: PASS (342.92 s)
 (5/5) repeat5.Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.ksm_ksmtuned.q35: STARTED
 (5/5) repeat5.Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.ksm_ksmtuned.q35: PASS (293.97 s)
RESULTS    : PASS 5 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0

mcasquer · 2024-02-08T10:51:11Z

@yanan-fu could you review this PR? Thanks !

yanan-fu · 2024-02-19T09:09:17Z

would we use the wait_for method to check the status before timeout ?

mcasquer · 2024-02-19T13:44:37Z

would we use the wait_for method to check the status before timeout ?

@yanan-fu IINM, the wait_for function waits for the process to finish, but the idea is to wait before starting the process (check KSM status). Is there a different way to do it?

yanan-fu · 2024-02-20T02:15:00Z

would we use the wait_for method to check the status before timeout ?

@yanan-fu IINM, the wait_for function waits for the process to finish, but the idea is to wait before starting the process (check KSM status). Is there a different way to do it?

Not exactly, it is wait for get the expected result instead of process to finish. ksm status is running is the check point.

mcasquer · 2024-02-20T09:33:18Z

would we use the wait_for method to check the status before timeout ?

@yanan-fu IINM, the wait_for function waits for the process to finish, but the idea is to wait before starting the process (check KSM status). Is there a different way to do it?

Not exactly, it is wait for get the expected result instead of process to finish. ksm status is running is the check point.

Sorry, it seems there are two wait_for functions 😀

mcasquer · 2024-02-20T09:41:27Z

Test passed with the new changes

 (1/1) Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.ksm_ksmtuned.q35: STARTED
 (1/1) Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.ksm_ksmtuned.q35: PASS (308.47 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0

mcasquer · 2024-02-20T09:42:00Z

@yanan-fu could you review again this PR? Thanks !

mcasquer · 2024-02-26T08:37:16Z

@yanan-fu this is a kindly reminder, could you review again this PR? Thanks !

qemu/tests/ksm_ksmtuned.py

mcasquer · 2024-02-27T07:13:48Z

 (1/1) Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.ksm_ksmtuned.q35: STARTED
 (1/1) Host_RHEL.m9.u4.ovmf.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.x86_64.io-github-autotest-qemu.ksm_ksmtuned.q35: PASS (279.51 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0

yanan-fu

Ack

yanan-fu · 2024-02-28T02:37:27Z

Call for multi-arch review, @PaulYuuu @fbq815 @MiriamDeng , thanks ~

MiriamDeng · 2024-02-28T07:36:10Z

It seems there's error as below on ppc64le
(2/2) Host_RHEL.m8.u10.product_rhel.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.ppc64le.io-github-autotest-qemu.ksm_ksmtuned: STARTED
(2/2) Host_RHEL.m8.u10.product_rhel.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.ppc64le.io-github-autotest-qemu.ksm_ksmtuned: ERROR: QEMU used memory doesn't reach 0.95 of guest mem 37888.0M in 1800s (2242.90 s)
[stdlog] 2024-02-28 08:53:22,934 aexpect.client DEBUG| [10.16.215.210] Sending command: cd /home/stress-1.0.4/;./configure && make install
[stdlog] 2024-02-28 08:53:34,545 aexpect.client DEBUG| [10.16.215.210] Sending command: echo $?
[stdlog] 2024-02-28 08:53:34,647 aexpect.client DEBUG| [10.16.215.210] Sending command (safe): cd /home/stress-1.0.4/
[stdlog] 2024-02-28 08:53:34,750 avocado.virttest.utils_test INFO | Launch stress with command: nohup stress --cpu 4 --io 4 --vm 2 --vm-bytes 18944M > /dev/null &
[stdlog] 2024-02-28 08:53:34,750 aexpect.client DEBUG| [10.16.215.210] Sending command (safe): nohup stress --cpu 4 --io 4 --vm 2 --vm-bytes 18944M > /dev/null &
[stdlog] 2024-02-28 08:53:37,357 avocado.virttest.utils_misc DEBUG| wait for stress app to start (2.002047 secs)
[stdlog] 2024-02-28 08:53:37,357 aexpect.client DEBUG| [10.16.215.210] Sending command: pidof -s stress
[stdlog] 2024-02-28 08:53:46,887 aexpect.client DEBUG| [10.16.215.210] Sending command: echo $?
[stdlog] 2024-02-28 09:23:50,733 avocado.utils.process INFO | Running 'systemctl restart ksmtuned.service'
[stdlog] 2024-02-28 09:23:50,767 avocado.utils.process INFO | Command 'systemctl restart ksmtuned.service' finished with 0 after 0.024247634s
[stdlog] 2024-02-28 09:23:50,985 avocado.test ERROR|
[stdlog] 2024-02-28 09:23:50,985 avocado.test ERROR| Reproduced traceback from: /usr/local/lib/python3.6/site-packages/avocado_framework_plugin_vt-103.0-py3.6.egg/avocado_vt/test.py:275
[stdlog] 2024-02-28 09:23:51,001 avocado.test ERROR| Traceback (most recent call last):
[stdlog] 2024-02-28 09:23:51,001 avocado.test ERROR| File "/root/avocado/data/avocado-vt/virttest/test-providers.d/downloads/io-github-autotest-qemu/qemu/tests/ksm_ksmtuned.py", line 136, in run
[stdlog] 2024-02-28 09:23:51,001 avocado.test ERROR| check_ksm(guest_mem, threshold_reached=True)
[stdlog] 2024-02-28 09:23:51,001 avocado.test ERROR| File "/root/avocado/data/avocado-vt/virttest/test-providers.d/downloads/io-github-autotest-qemu/qemu/tests/ksm_ksmtuned.py", line 84, in check_ksm
[stdlog] 2024-02-28 09:23:51,001 avocado.test ERROR| "%ss" % (mem_thres, mem // 1024, stress_timeout))
[stdlog] 2024-02-28 09:23:51,001 avocado.test ERROR| File "/usr/local/lib/python3.6/site-packages/avocado_framework-100.1-py3.6.egg/avocado/core/test.py", line 779, in wrapper
[stdlog] 2024-02-28 09:23:51,001 avocado.test ERROR| return func(actual_message)
[stdlog] 2024-02-28 09:23:51,001 avocado.test ERROR| File "/usr/local/lib/python3.6/site-packages/avocado_framework-100.1-py3.6.egg/avocado/core/test.py", line 809, in error
[stdlog] 2024-02-28 09:23:51,001 avocado.test ERROR| raise exceptions.TestError(msg)
[stdlog] 2024-02-28 09:23:51,001 avocado.test ERROR| avocado.core.exceptions.TestError: QEMU used memory doesn't reach 0.95 of guest mem 37888.0M in 1800s
[stdlog] 2024-02-28 09:23:51,001 avocado.test ERROR|

yanan-fu · 2024-02-28T10:13:27Z

It seems there's error as below on ppc64le (2/2) Host_RHEL.m8.u10.product_rhel.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.ppc64le.io-github-autotest-qemu.ksm_ksmtuned: STARTED (2/2) Host_RHEL.m8.u10.product_rhel.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.ppc64le.io-github-autotest-qemu.ksm_ksmtuned: ERROR: QEMU used memory doesn't reach 0.95 of guest mem 37888.0M in 1800s (2242.90 s)

Logically, it is not related with this patch, it is the mem_thres defined in the script and qemu does not reach it after stress i think.

mcasquer · 2024-02-28T10:22:56Z

Logically, it is not related with this patch, it is the mem_thres defined in the script and qemu does not reach it after stress i think.

That's it, I am debugging the case on @MiriamDeng environment but for sure it is not related with the current patch. So that makes me think about how long is being ksm_ksmtuned failing on ppc... 🤔 😅

mcasquer · 2024-02-28T10:24:52Z

Logically, it is not related with this patch, it is the mem_thres defined in the script and qemu does not reach it after stress i think.

That's it, I am debugging the case on @MiriamDeng environment but for sure it is not related with the current patch. So that makes me think about how long is being ksm_ksmtuned failing on ppc... 🤔 😅

Furthermore, the test case takes ~5 minutes on x86_64 but more than 30 minutes in ppc

After checking the timings, the case failed because ksmtuned needs some more time in order to start ksm after reaching the threshold. Signed-off-by: mcasquer <[email protected]>

mcasquer · 2024-03-04T06:25:34Z

@fbq815 @MiriamDeng this is a kindly reminder, could you review this PR? Thanks !

fbq815 · 2024-03-04T15:23:47Z

Test result on s390x:
RHEL9:
ksm_ksmtuned.s390-virtio: PASS (158.85 s)
RHEL8:
ksm_ksmtuned.s390-virtio: PASS (177.00 s)
LGTM

MiriamDeng · 2024-03-05T04:23:52Z

ACK
(1/2) Host_RHEL.m8.u10.product_rhel.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.ppc64le.io-github-autotest-qemu.unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads: STARTED
(1/2) Host_RHEL.m8.u10.product_rhel.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.ppc64le.io-github-autotest-qemu.unattended_install.cdrom.extra_cdrom_ks.default_install.aio_threads: PASS (2061.82 s)
(2/2) Host_RHEL.m8.u10.product_rhel.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.ppc64le.io-github-autotest-qemu.ksm_ksmtuned: STARTED
(2/2) Host_RHEL.m8.u10.product_rhel.qcow2.virtio_scsi.up.virtio_net.Guest.RHEL.9.4.0.ppc64le.io-github-autotest-qemu.ksm_ksmtuned: PASS (385.82 s)

mcasquer marked this pull request as ready for review February 8, 2024 10:50

mcasquer force-pushed the 1932_ksmtuned branch from 53fc43a to 2ee4bb4 Compare February 20, 2024 09:32

mcasquer force-pushed the 1932_ksmtuned branch 3 times, most recently from 2de201a to 1667510 Compare February 20, 2024 09:39

mcasquer force-pushed the 1932_ksmtuned branch 2 times, most recently from d592fde to 2225981 Compare February 26, 2024 08:36

yanan-fu reviewed Feb 27, 2024

View reviewed changes

qemu/tests/ksm_ksmtuned.py Outdated Show resolved Hide resolved

mcasquer force-pushed the 1932_ksmtuned branch 2 times, most recently from 0f9e04e to fe657fc Compare February 27, 2024 07:11

mcasquer force-pushed the 1932_ksmtuned branch 4 times, most recently from f4199ac to cd417de Compare February 27, 2024 11:29

yanan-fu approved these changes Feb 28, 2024

View reviewed changes

PaulYuuu approved these changes Feb 29, 2024

View reviewed changes

ksm_ksmtuned: gives more time for ksm to start

5af223d

After checking the timings, the case failed because ksmtuned needs some more time in order to start ksm after reaching the threshold. Signed-off-by: mcasquer <[email protected]>

mcasquer force-pushed the 1932_ksmtuned branch from cd417de to 5af223d Compare March 4, 2024 06:25

fbq815 approved these changes Mar 4, 2024

View reviewed changes

PaulYuuu merged commit abc55ea into autotest:master Mar 5, 2024
6 checks passed

mcasquer deleted the 1932_ksmtuned branch March 5, 2024 06:44

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ksm_ksmtuned: gives more time for ksm to start #3957

ksm_ksmtuned: gives more time for ksm to start #3957

mcasquer commented Feb 8, 2024

mcasquer commented Feb 8, 2024

mcasquer commented Feb 8, 2024

yanan-fu commented Feb 19, 2024

mcasquer commented Feb 19, 2024

yanan-fu commented Feb 20, 2024

mcasquer commented Feb 20, 2024

mcasquer commented Feb 20, 2024

mcasquer commented Feb 20, 2024

mcasquer commented Feb 26, 2024

mcasquer commented Feb 27, 2024

yanan-fu left a comment

yanan-fu commented Feb 28, 2024

MiriamDeng commented Feb 28, 2024 •

edited

Loading

yanan-fu commented Feb 28, 2024

mcasquer commented Feb 28, 2024

mcasquer commented Feb 28, 2024

mcasquer commented Mar 4, 2024

fbq815 commented Mar 4, 2024

MiriamDeng commented Mar 5, 2024

ksm_ksmtuned: gives more time for ksm to start #3957

ksm_ksmtuned: gives more time for ksm to start #3957

Conversation

mcasquer commented Feb 8, 2024

mcasquer commented Feb 8, 2024

mcasquer commented Feb 8, 2024

yanan-fu commented Feb 19, 2024

mcasquer commented Feb 19, 2024

yanan-fu commented Feb 20, 2024

mcasquer commented Feb 20, 2024

mcasquer commented Feb 20, 2024

mcasquer commented Feb 20, 2024

mcasquer commented Feb 26, 2024

mcasquer commented Feb 27, 2024

yanan-fu left a comment

Choose a reason for hiding this comment

yanan-fu commented Feb 28, 2024

MiriamDeng commented Feb 28, 2024 • edited Loading

yanan-fu commented Feb 28, 2024

mcasquer commented Feb 28, 2024

mcasquer commented Feb 28, 2024

mcasquer commented Mar 4, 2024

fbq815 commented Mar 4, 2024

MiriamDeng commented Mar 5, 2024

MiriamDeng commented Feb 28, 2024 •

edited

Loading