Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker executor cannot recognize the volumes option. #88

Open
danielecook opened this issue Nov 1, 2023 · 0 comments
Open

Docker executor cannot recognize the volumes option. #88

danielecook opened this issue Nov 1, 2023 · 0 comments

Comments

@danielecook
Copy link
Contributor

I am trying to get a local setup running for debugging purposes. Data will be staged locally to reduce latency. Unfortunately, the volumes option for the docker executor does not appear to work.

Current config

[executors.docker]
type = docker
max_workers = 20
image = ####
scratch = /tmp/scratch
include_aws_env = False
volumes = [["/home", "/home"]]
interactive = False

The get_docker_executor_config does not retrieve volumes currently.

def get_docker_executor_config(config: SectionProxy) -> SectionProxy:
"""
Returns a config for DockerExecutor.
"""
keys = ["image", "scratch", "job_monitor_interval", "vcpus", "gpus", "memory"]
return create_config_section({key: config[key] for key in keys if key in config})

It may be helpful to throw an error for additional options specified here that cannot be used with the Docker executor.

Additionally, I wonder whether mounting the workflow working directory would be beneficial?

self._default_job_options: Dict[str, Any] = {
"vcpus": config.getint("vcpus", fallback=1),
"gpus": config.getint("gpus", fallback=0),
"memory": config.getint("memory", fallback=4),
"shared_memory": config.getint("shared_memory", fallback=None),
"interactive": config.getboolean("interactive", fallback=False),
"include_aws_env": config.getboolean("include_aws_env", fallback=True),
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant