[RFC] teuthology: add email headers for suite/user

[batrick]

For advanced filtering/organization.

Fixes: https://tracker.ceph.com/issues/64839

[batrick]

I'm probably doing something bone-headed but can anyone explain why my tests for this PR all result in a dead job:

https://pulpito.ceph.com/pdonnell-2024-03-12_15:56:41-teuthology:nop-main-distro-default-smithi/

$ teuthology-suite -t email-qa-headers --machine-type smithi --email [email protected] --suite teuthology:nop --ceph main --ceph-repo https://github.com/ceph/ceph.git --suite-repo https://github.com/ceph/ceph.git -n 100 -p 1 --force-priority --limit 1
2024-03-12 15:56:41,936.936 INFO:teuthology.suite:Using random seed=949
2024-03-12 15:56:41,937.937 INFO:teuthology.suite.run:kernel sha1: distro
2024-03-12 15:56:42,515.515 INFO:teuthology.suite.run:ceph sha1: f73472cd7e55597d97b9164e33be8a20ccadcd27
2024-03-12 15:56:42,515.515 INFO:teuthology.suite.run:skipping ceph package verification
2024-03-12 15:56:42,515.515 INFO:teuthology.suite.run:ceph branch: main f73472cd7e55597d97b9164e33be8a20ccadcd27
2024-03-12 15:56:42,576.576 INFO:teuthology.repo_utils:Fetching github.com_ceph_ceph_main from origin
2024-03-12 15:56:49,222.222 INFO:teuthology.suite.run:teuthology branch: email-qa-headers 3660aba7a8f4640836095d832fa38eb65e875815
2024-03-12 15:56:49,278.278 INFO:teuthology.suite.run:Suite teuthology:nop in /cephfs/home/pdonnell/src/github.com_ceph_ceph_main/qa/suites/teuthology/nop generated 2 jobs (not yet filtered or merged)
2024-03-12 15:56:49,441.441 INFO:teuthology.suite.util:Container build incomplete
2024-03-12 15:56:49,442.442 ERROR:teuthology.suite.run:Packages for os_type 'centos', flavor default and ceph hash 'f73472cd7e55597d97b9164e33be8a20ccadcd27' not found
2024-03-12 15:56:50,888.888 INFO:teuthology.suite.run:Stopped after 1 jobs due to --limit=1
2024-03-12 15:56:50,888.888 INFO:teuthology.suite.run:--newest supplied, backtracked 1 commits to d7353ad4342f698eae25524af998101e3c847e52
Job scheduled with name pdonnell-2024-03-12_15:56:41-teuthology:nop-main-distro-default-smithi and ID 7594597
2024-03-12 15:56:51,964.964 INFO:teuthology.suite.run:Scheduling teuthology:nop/{clusters/empty tasks/nop}
Job scheduled with name pdonnell-2024-03-12_15:56:41-teuthology:nop-main-distro-default-smithi and ID 7594598
2024-03-12 15:56:53,004.004 INFO:teuthology.suite.run:Suite teuthology:nop in /cephfs/home/pdonnell/src/github.com_ceph_ceph_main/qa/suites/teuthology/nop scheduled 1 jobs.
2024-03-12 15:56:53,004.004 INFO:teuthology.suite.run:1/2 jobs were filtered out.
2024-03-12 15:56:53,004.004 INFO:teuthology.suite.run:Scheduled 1 jobs in total.
Job scheduled with name pdonnell-2024-03-12_15:56:41-teuthology:nop-main-distro-default-smithi and ID 7594599
2024-03-12 15:56:53,993.993 INFO:teuthology.suite.run:Test results viewable at https://pulpito.ceph.com/pdonnell-2024-03-12_15:56:41-teuthology:nop-main-distro-default-smithi/

[VallariAg]

Missing "(" in this line is causing the dead job. I found this by running this in teuthology machine:

vallariag@teuthology:~$ cd /a/pdonnell-2024-03-12_15:56:41-teuthology:nop-main-distro-default-smithi/7594598
vallariag@teuthology:/a/pdonnell-2024-03-12_15:56:41-teuthology:nop-main-distro-default-smithi/7594598$ ls
orig.config.yaml  supervisor.7594598.log
vallariag@teuthology:/a/pdonnell-2024-03-12_15:56:41-teuthology:nop-main-distro-default-smithi/7594598$ cat supervisor.7594598.log
2024-03-12T16:02:17.089 INFO:root:teuthology version: 0.0.1.dev263+g3660aba
2024-03-12T16:02:17.089 INFO:teuthology.dispatcher.supervisor:Running job 7594598
2024-03-12T16:02:17.089 DEBUG:teuthology.dispatcher.supervisor:Running: /home/teuthworker/src/git.ceph.com_teuthology_3660aba7a8f4640836095d832fa38eb65e875815/virtualenv/bin/teuthology -v --owner scheduled_pdonnell@teuthology --archive /home/teuthworker/archive/pdonnell-2024-03-12_15:56:41-teuthology:nop-main-distro-default-smithi/7594598 --name pdonnell-2024-03-12_15:56:41-teuthology:nop-main-distro-default-smithi --description teuthology:nop/{clusters/empty tasks/nop} -- /home/teuthworker/archive/pdonnell-2024-03-12_15:56:41-teuthology:nop-main-distro-default-smithi/7594598/orig.config.yaml
2024-03-12T16:02:17.092 INFO:teuthology.dispatcher.supervisor:Job archive: /home/teuthworker/archive/pdonnell-2024-03-12_15:56:41-teuthology:nop-main-distro-default-smithi/7594598
2024-03-12T16:02:17.093 INFO:teuthology.dispatcher.supervisor:Job PID: 3302657
2024-03-12T16:02:17.093 INFO:teuthology.dispatcher.supervisor:Running with watchdog
2024-03-12T16:04:17.191 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/
2024-03-12T16:04:17.277 ERROR:teuthology.dispatcher.supervisor:Child exited with code 1
vallariag@teuthology:/a/pdonnell-2024-03-12_15:56:41-teuthology:nop-main-distro-default-smithi/7594598$ /home/teuthworker/src/git.ceph.com_teuthology_3660aba7a8f4640836095d832fa38eb65e875815/virtualenv/bin/teuthology --help
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_teuthology_3660aba7a8f4640836095d832fa38eb65e875815/virtualenv/bin/teuthology", line 5, in <module>
    from scripts.run import main
  File "/home/teuthworker/src/git.ceph.com_teuthology_3660aba7a8f4640836095d832fa38eb65e875815/scripts/run.py", line 33, in <module>
    import teuthology.run
  File "/home/teuthworker/src/git.ceph.com_teuthology_3660aba7a8f4640836095d832fa38eb65e875815/teuthology/run.py", line 14, in <module>
    from teuthology.results import email_results
  File "/home/teuthworker/src/git.ceph.com_teuthology_3660aba7a8f4640836095d832fa38eb65e875815/teuthology/results.py", line 78
    sender = config.results_sending_email or 'teuthology')
                                                         ^
SyntaxError: unmatched ')'

[batrick]

Thank you! I knew it:

I'm probably doing something bone-headed

[batrick]

For the record, I'm not working on this anymore. I think someone was interested in finishing it? @zmc ?

Edit: using paddles queries (IIRC)?

[zmc]

ah I think what I was saying when this came up was that one approach could be querying paddles for the run, and getting the user and email values from the response body.

A possibly better approach could be to write out a small metadata file in the run's archive directory (as opposed to the job subdirs), called e.g. run.yml, that could contain some relevant information like user, suite, subset, etc. It would be appropriate to then add a run_info() method to ResultsSerializer.

Of course you'd want to check for the existence of the file before writing it, as writing multiple copies of the same content wouldn't be helpful.

[batrick]

ah I think what I was saying when this came up was that one approach could be querying paddles for the run, and getting the user and email values from the response body.

A possibly better approach could be to write out a small metadata file in the run's archive directory (as opposed to the job subdirs), called e.g. run.yml, that could contain some relevant information like user, suite, subset, etc. It would be appropriate to then add a run_info() method to ResultsSerializer.

Of course you'd want to check for the existence of the file before writing it, as writing multiple copies of the same content wouldn't be helpful.

My understanding during our standup discussion was that querying paddles would be preferable, no?

[kshtsk]

Only querying paddles must be done outside of the teuthology run, maybe in dispatcher/supervisor?