Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server tests : more pythonic process management; fix bare except: #6146

Merged
merged 9 commits into from
Mar 20, 2024

Conversation

cebtenzzre
Copy link
Collaborator

@cebtenzzre cebtenzzre commented Mar 18, 2024

ref #6098 (comment)

This code has been consistently confusing to me. Here I suggest a few simplifications:

  • Don't add an extra \n to a random selection of print()s. A habit from C perhaps? Let me know if any (or all) of these are intentional - it just looked strange to me to have a blank line after so many prints.
  • I changed the process management code in the server tests to use built-in subprocess functions instead of os.kill or psutil.
  • Before we were sending SIGINT, waiting 50ms, then sending SIGKILL in a loop, and after several tries also sending SIGKILL with both native Python and psutil at the same time. This seems really odd to me, so I changed it to just send SIGKILL if SIGINT didn't kill the process after 500ms. Maybe Popen.terminate() was intended as the second stage of killing, but this will have the same effect as SIGKILL for the server process, even on Unix, as it is not explicitly handled via sigaction.
  • I also fixed the neighboring except: clause - it's wrong to ignore exceptions like KeyboardInterrupt and SystemExit, so I changed it to except Exception:. And traceback.print_exc seemed more appropriate here.

@cebtenzzre cebtenzzre requested a review from phymbert March 18, 2024 18:39
Copy link
Collaborator

@phymbert phymbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably better code but the build failed... maybe you must test it before

@cebtenzzre
Copy link
Collaborator Author

cebtenzzre commented Mar 19, 2024

I was able to run the tests locally after making the connect routine match what the server does - it uses the first address family returned by getaddrinfo, whether it is ipv4 or ipv6. Otherwise, the connection failed because the server was listening on ::1 and the client was trying to connect to 127.0.0.1.

see also #5372 (comment)

@cebtenzzre cebtenzzre requested a review from phymbert March 19, 2024 15:40
@phymbert
Copy link
Collaborator

Otherwise, the connection failed because the server was listening on ::1 and the client was trying to connect to 127.0.0.1.

There is a hack on windows server test to force the server to listen 0.0.0.0. But it's probably not the same issue you faced with ipv6

Copy link
Collaborator

@phymbert phymbert left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking into this, your contribution on server tests is highly appreciated 👍
If you don't mind I will request your review on changes related to python server tests.

I have included fixes for:

All tests are now working and the python code is clearer, great!

@phymbert phymbert merged commit bd60d82 into master Mar 20, 2024
26 checks passed
@phymbert phymbert deleted the ceb/pythonic-server-tests branch March 20, 2024 05:34
hodlen pushed a commit to hodlen/llama.cpp that referenced this pull request Apr 1, 2024
…gerganov#6146)

* server tests : remove seemingly redundant newlines in print()

* server tests : use built-in subprocess features, not os.kill and psutil

* server tests : do not catch e.g. SystemExit; use print_exc

* server tests: handle TimeoutExpired exception

* server tests: fix connect on dual-stack systems

* server: tests: add new tokens regex on windows generated following new repeat penalties default changed in (ggerganov#6127)

* server: tests: remove the hack on windows since now we get the good socket family

* server: tests: add new tokens regex following new repeat penalties default changed in (ggerganov#6127)

* server: tests: add new tokens regex following new repeat penalties default changed in (ggerganov#6127)

---------

Co-authored-by: Pierrick HYMBERT <[email protected]>
hodlen pushed a commit to hodlen/llama.cpp that referenced this pull request Apr 3, 2024
…gerganov#6146)

* server tests : remove seemingly redundant newlines in print()

* server tests : use built-in subprocess features, not os.kill and psutil

* server tests : do not catch e.g. SystemExit; use print_exc

* server tests: handle TimeoutExpired exception

* server tests: fix connect on dual-stack systems

* server: tests: add new tokens regex on windows generated following new repeat penalties default changed in (ggerganov#6127)

* server: tests: remove the hack on windows since now we get the good socket family

* server: tests: add new tokens regex following new repeat penalties default changed in (ggerganov#6127)

* server: tests: add new tokens regex following new repeat penalties default changed in (ggerganov#6127)

---------

Co-authored-by: Pierrick HYMBERT <[email protected]>
tybalex pushed a commit to rubra-ai/tools.cpp that referenced this pull request Apr 17, 2024
…gerganov#6146)

* server tests : remove seemingly redundant newlines in print()

* server tests : use built-in subprocess features, not os.kill and psutil

* server tests : do not catch e.g. SystemExit; use print_exc

* server tests: handle TimeoutExpired exception

* server tests: fix connect on dual-stack systems

* server: tests: add new tokens regex on windows generated following new repeat penalties default changed in (ggerganov#6127)

* server: tests: remove the hack on windows since now we get the good socket family

* server: tests: add new tokens regex following new repeat penalties default changed in (ggerganov#6127)

* server: tests: add new tokens regex following new repeat penalties default changed in (ggerganov#6127)

---------

Co-authored-by: Pierrick HYMBERT <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants