-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
dsc failed to start a downstairs #1498
Comments
In dsc/src/main.rs, we have the following code that starts a downstairs:
In our logs above, we see the first message |
I was able to reproduce it outside of CI using a debug build. The stacks look mostly similar, but this one stuck out:
|
I believe this was falsely closed because you included the phrase |
Another case of this same issue in CI: https://buildomat.eng.oxide.computer/wg/0/details/01JBM7SARCEJ2J41N4BHD5BAW3/TjTXDhs9Rfb6bubRvYJRhVjoHcZbPNeUbNPHhKcjyKExIrUe/01JBM7WS8Q1A29N2QQPV5RPM6N This time it was during the |
Here is another dsc, stuck in the same place (a task waiting for one of the downstairs to start when that downstairs has already started):
FD 12 appears to be the output file where the downstairs should be sending output:
I'm not sure if this thread in the same dsc stack is important or not, but it does not appear to be changing:
From pfiles, fd 3 is:
|
In CI (link may be bad) job: https://buildomat.eng.oxide.computer/wg/0/artefact/01J9KZDBT7Q76BMAZ3NFT2EM6G/JHuMk44VV03fuovquSYDTdL9uCEruARhbYRD7WhRUTif7Lam/01J9KZDV6QXGVBQG0ZHA6JNXTH/01J9M3TRMJHBD7KTGG9QWWK022/dsc-out.txt
We had the test timeout.
Looking at test log output, we can see a new check we added is waiting for all the downstairs to respond:
The waiting for it waits till the test hits a timeout and we abort it.
In the logs for
dsc
we can see it got toStarting
on client 0, but never toRunning
:A third bit of information, if I look at the output for client 0, port 8810, I do see that it started:
This suggests the problem is somewhere in the messages between different parts of
dsc
.The text was updated successfully, but these errors were encountered: