Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement a '--backend' flag to run a server in the background #2

Open
anko opened this issue May 5, 2020 · 1 comment
Open

Implement a '--backend' flag to run a server in the background #2

anko opened this issue May 5, 2020 · 1 comment
Assignees

Comments

@anko
Copy link
Owner

anko commented May 5, 2020

Why?

This would be helpful for testing some types of programs:

  • Programs that require a long-running server process to be running in the background, like web services.
  • Interpreters that have a relatively long start up time. These could gain performance by feeding test code into a single process and evaling it, rather than starting one instance of the interpreter per test.

Current workaround

Use --jobs 1, and in your !program, start the backend if TXM_INDEX equals TXM_INDEX_FIRST, or stop it if TXM_INDEX equals TXM_INDEX_LAST.

For any other value of --jobs, this workaround introduces a race condition due to non-deterministic ordering.

How it could work

Before running the first test, start the program specified as --backend="program goes here" in the background. After the last test, terminate it.

If the backend program exits at any point, fail all so-far-incomplete tests. In the YAML for the failures, give the backend program's text, stdout, stderr, and exit code.

Limitations

Q: How do we know when the backend is truly "up" and ready to receive the first test?
A: We can't; it's application specific. We should document that it's the user's program's responsibility to deal with it.

@anko anko self-assigned this May 5, 2020
@anko
Copy link
Owner Author

anko commented May 11, 2020

Good news: Implemented in the backend-flag branch. Tests work locally.

Bad news: Fails on Travis, hanging without explanation on the first associated test. Theories:

  • Is waitOn blocking forever?

    → No. It's given a timeout option so it should throw.

    txm/test.ls

    Lines 1446 to 1454 in 63f2baa

    // Wait for up to a second the server to start, then fetch the URI
    // and print it.
    waitOn({ resources: [ uri ], timeout: 1000 }, (error) => {
    if (error) throw error
    http.get(uri, (res) => {
    res.pipe(process.stdout)
    res.on('end', () => { process.stdout.write('\\n') })
    })
    })

  • The server is binding to an IPv6 interface but the client is hitting IPv4 or vice-versa?

    → No. The server binds to localhost:8888,

    txm/test.ls

    Line 1481 in 63f2baa

    .listen(8888, 'localhost')
    and the client reads from localhost. Even if Travis has localhost set to something insane, waitOn should throw when it can't find anything.

Needs more investigation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant