-
-
Notifications
You must be signed in to change notification settings - Fork 371
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Clean up Jvm.*
subprocess/inprocess APIs (500USD Bounty)
#3772
Comments
lihaoyi
changed the title
Clean up
Clean up Oct 24, 2024
Jvm.*
subprocess/inprocess APIs Jvm.*
subprocess/inprocess APIs (500USD Bounty)
vishwamartur
added a commit
to vishwamartur/mill
that referenced
this issue
Nov 16, 2024
Related to com-lihaoyi#3772 Refactor `Jvm.scala` to consolidate subprocess and classloader spawning operations into four specified signatures. * **Refactor `callSubprocess` method:** - Rename to `call`. - Update parameters to match the specified `call` signature. - Use `jvmCommandArgs` to generate command arguments. - Call `os.call` with the updated parameters. * **Refactor `runSubprocess` method:** - Rename to `spawn`. - Update parameters to match the specified `spawn` signature. - Use `jvmCommandArgs` to generate command arguments. - Call `os.spawn` with the updated parameters. * **Add `spawnClassloader` method:** - Create a new method to match the specified `spawnClassloader` signature. - Use `mill.api.ClassLoader.create` to create a classloader. * **Add `callClassloader` method:** - Create a new method to match the specified `callClassloader` signature. - Use `spawnClassloader` to create a classloader and set it as the context classloader. - Execute the provided function with the new classloader and restore the old classloader afterward. * **Add tests in `JvmTests.scala`:** - Add tests for the new `call` method. - Add tests for the new `spawn` method. - Add tests for the new `callClassloader` method. - Add tests for the new `spawnClassloader` method.
lihaoyi
added a commit
that referenced
this issue
Dec 9, 2024
…4085) Fixes #657 We re-use `MillBackgroundWrapper` and extend it to optionally spawn subprocess instead of calling the in-process main method. There's a bit of overhead in the JVM wrapper, but we need some kind of process wrapper to manage the mutex and termination even when the Mill process terminates, and eventually when #4007 lands we can use that to provide lighter-weight wrappers. We add a shutdown hook and copy the termination grace-period logic from OS-Lib to try and gently kill the wrapped process when necessary Integrated this into `PythonModule#runBackground` and added a unit test to verify the asynchronous launch, background existence (by checking that a file lock is taken) and termination on deletion. We can integrate this into `TypescriptModule` as well but punting that for later The subprocess APIs are a mess but leaving that to fix in 0.13.0 #3772
As part of this we should move |
jodersky
pushed a commit
to jodersky/mill
that referenced
this issue
Jan 14, 2025
…om-lihaoyi#4085) Fixes com-lihaoyi#657 We re-use `MillBackgroundWrapper` and extend it to optionally spawn subprocess instead of calling the in-process main method. There's a bit of overhead in the JVM wrapper, but we need some kind of process wrapper to manage the mutex and termination even when the Mill process terminates, and eventually when com-lihaoyi#4007 lands we can use that to provide lighter-weight wrappers. We add a shutdown hook and copy the termination grace-period logic from OS-Lib to try and gently kill the wrapped process when necessary Integrated this into `PythonModule#runBackground` and added a unit test to verify the asynchronous launch, background existence (by checking that a file lock is taken) and termination on deletion. We can integrate this into `TypescriptModule` as well but punting that for later The subprocess APIs are a mess but leaving that to fix in 0.13.0 com-lihaoyi#3772
Is this still valid, I would like to work on it this weekend. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
From the maintainer Li Haoyi: I'm putting a 500USD bounty on this issue, payable by bank transfer on a merged PR implementing this.
The various
Jvm.*
methods are a mess. #3770 makes them marginally more messy, but they've been messy for years and could use a cleanup.The goal of this issue is to consolidate all subprocess and classloader spawning operations in
Jvm.scala
into (approximately) the following four signaturesJvm.spawn
andJvm.call
intentionally follow the signatures ofos.spawn
andos.call
.Jvm.spawnClassloader
andJvm.callClassloader
are in-memory variations of the theme, with no subprocess-related parameters and an API tweaked to work in-memory.Jvm.scala
should be refactored to go through the four APIs above. The other APIs should all be deprecated.Jvm.scala
spawning subprocesses or classloaders should go through those four APIs,The text was updated successfully, but these errors were encountered: