[utilities] catch SoSTimeoutError in sos_get_command_output

[pmoravec]

When calling collect_cmd_output and timeout is hit, SoSTimeoutError is raised and never caught (until in plugin wrapper).

Resolves: #3331

---

Please place an 'X' inside each '[]' to confirm you adhere to our Contributor Guidelines

• Is the commit message split over multiple lines and hard-wrapped at 72 characters?
• Is the subject and message clear and concise?
• Does the subject start with [plugin_name] if submitting a plugin patch or a [section_name] if part of the core sosreport code?
• Does the commit contain a Signed-off-by: First Lastname [email protected]?
• Are any related Issues or existing PRs properly referenced via a Closes (Issue) or Resolved (PR) line?

[pmoravec]

Reproducer: on a system without virsh, run:

echo "sleep 10" > ~/bin/virsh
chmod a+x ~/bin/virsh
sos report -o virsh --cmd-timeout=3 --batch --build

which reports:

caught exception in plugin method "virsh.setup()"
writing traceback to sos_logs/virsh-plugin-errors.txt

with content:

Traceback (most recent call last):
  File "/root/sos-main/sos/report/__init__.py", line 1232, in setup
    plug.setup()
  File "/root/sos-main/sos/report/plugins/virsh.py", line 54, in setup
    foreground=True)
  File "/root/sos-main/sos/report/plugins/__init__.py", line 2534, in collect_cmd_output
    subdir=subdir, tags=tags
  File "/root/sos-main/sos/report/plugins/__init__.py", line 2370, in _collect_cmd_output
    to_file=out_file
  File "/root/sos-main/sos/utilities.py", line 267, in sos_get_command_output
    raise e
  File "/root/sos-main/sos/utilities.py", line 236, in sos_get_command_output
    _check_poller(p)
  File "/root/sos-main/sos/utilities.py", line 186, in _check_poller
    raise SoSTimeoutError
sos.utilities.SoSTimeoutError

The exception is raised directly in sos_get_command_output in _check_poller sub-routine.

[packit-as-a-service]

Congratulations! One of the builds has completed. 🍾

You can install the built RPMs by following these steps:

• sudo yum install -y dnf-plugins-core on RHEL 8
• sudo dnf install -y dnf-plugins-core on Fedora
• dnf copr enable packit/sosreport-sos-3331
• And now you can install the packages.

Please note that the RPMs should be used only in a testing environment.

[pmoravec]

The patch should rather be like:

@@ -232,8 +232,16 @@ def sos_get_command_output(command, time
             reader = FakeReader(p, binary)
 
         if poller:
-            while reader.running:
-                _check_poller(p)
+            try:
+                while reader.running:
+                    _check_poller(p)
+            except SoSTimeoutError:
+                p.terminate()
+                if to_file:
+                    _output.close()
+                reader.running = False
+                return {'status': 124, 'output': reader.get_contents(),
+                        'truncated': reader.is_full}
         else:
             try:
                 # override timeout=0 to timeout=None, as Popen will treat the

but even then, sos gets stuck forever before shutdown, with many threads pending on sos/utilities.py, line 484 (read from pipe of stuck virsh command - I can reproduce this with a real frozing virsh command only, not by mocked sleep) . We might add a poller to the read from pipe, but.. isn't it generally fragile and overkill?

[arif-ali]

@pmoravec still need this to go through?

[pmoravec]

Well the reproducer is still valid on current upstream and I still dont have an idea of a proper fix /o.