Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ERROR] Could not find auth config for quay.io. Returning empty builder #386

Open
mepstein-gh opened this issue Nov 16, 2017 · 11 comments
Open

[ERROR] Could not find auth config for quay.io. Returning empty builder #386

mepstein-gh opened this issue Nov 16, 2017 · 11 comments

Comments

@mepstein-gh
Copy link

mepstein-gh commented Nov 16, 2017 •
edited
Loading

In attempting to run rabix (1.0.3) on the command line on a CWL file that uses a docker container on quay.io, I got the above error message. Attached is a zip with a CWL file, an inputs file, and a log of the run. (I tried this with rabix 1.0.1, and it worked. In that case, that message also appeared, but the job ran without failure.)
KNF.zip
@milos-ljubinkovic
Copy link
Contributor

Sorry for that, it seems that there are some issues between this image and our new permission system. When executor.set_permissions=true is present in config the current system user (or the value of executor.permission.uid) is set as the user in docker container. You can get this task to execute on 1.0.3 by setting the set_permissions to false or setting the uuid to root or 0.

This is done so that output files will be owned by the current user and permission checks are supposed to be disabled inside the container but for some reason that didn't happen here. Are you using OSX? That could be a reason, and if you are using osx you can safely disable executor.set_permissions as docker for osx automatically handles permissions.

@mepstein-gh
Copy link
Author

mepstein-gh commented Nov 17, 2017 •
edited
Loading

I'm using linux (arch linux). I was able to change that config setting, and got it to run. I would like to have the output files owned by the current user, so hopefully this can be fixed in another way. Thanks.

@milos-ljubinkovic
Copy link
Contributor

We are investigating this issue and what can be changed. But it seems that there is another way to fix ownership issues on docker deamon level on your system.

You'd first edit your /etc/docker/daemon.json to contain the value: "userns-remap": "USER" where USER is your local username and adding following lines to your /etc/subuid and /etc/subgid files respectively:
USER:UID:1
USER:999:1
Where USER is again your username and UID is your unique id (output of id -u shell command).
(999 is there because it's the default docker group but this can be any other group id)

After you restart the docker deamon all files created while inside any container (not only rabix-created ones) will be remaped to be owned by you when inspected outside.

Because OSX, our standard development environment and cloud environments we target for execution have different permission/ownership handling than regular linux users we might default to suggesting linux users add this config. Not to mention that this method also fixes the same issue when using any docker container and not only in rabix.

Additional information about userns-remap can be found in this article: https://www.jujens.eu/posts/en/2017/Jul/02/docker-userns-remap/

@DavidAustinNix
Copy link

Seeing the same issue on MacOS 10.13.2 with a new install of Docker and Rabix 1.0.4. Example fails. Tried the executor.set_permissions=false modification of the config/core.properties without success.

MacBook-Pro-5:rabix-cli-1.0.4 u0028003$ ./rabix examples/dna2protein/dna2protein.cwl.json examples/dna2protein/inputs.json [2018-01-15 16:32:13.637] [INFO] Job root.Transcribe has started [2018-01-15 16:32:13.668] [INFO] Pulling docker image python:2-alpine [2018-01-15 16:32:13.675] [WARN] Could not find auth field for https://index.docker.io/v1/ [2018-01-15 16:32:14.411] [INFO] Running command line: python transcribe_argparse.py -d /Applications/BioApps/Rabix/rabix-cli-1.0.4/examples/dna2protein/data/input.txt --verbose > /Applications/BioApps/Rabix/rabix-cli-1.0.4/examples/dna2protein/dna2protein.cwl-2018-01-15-163212.28/root/Transcribe/rna.txt [2018-01-15 16:32:14.617] [WARN] Start container method timed-out but still started the container, recovering. [2018-01-15 16:32:15.678] [INFO] Job root.Transcribe has completed [2018-01-15 16:32:15.731] [INFO] Job root.Translate has started [2018-01-15 16:32:15.739] [INFO] Pulling docker image python:2-alpine [2018-01-15 16:32:15.739] [WARN] Could not find auth field for https://index.docker.io/v1/ [2018-01-15 16:32:15.752] [INFO] Running command line: python translate.py > /Applications/BioApps/Rabix/rabix-cli-1.0.4/examples/dna2protein/dna2protein.cwl-2018-01-15-163212.28/root/Translate/protein.txt [2018-01-15 16:32:15.863] [WARN] Start container method timed-out but still started the container, recovering. [2018-01-15 16:32:16.904] [INFO] Job root.Translate has completed { "output_protein" : null }

Thanks, David

@milos-ljubinkovic
Copy link
Contributor

milos-ljubinkovic commented Jan 16, 2018 •
edited
Loading

@DavidAustinNix
The key issue here is the line "[WARN] Start container method timed-out but still started the container, recovering. "

There is maybe something wrong with your docker server that caused the client to hang and not complete the execution. Which version of docker are you running and have you tested running docker images outside of rabix?

@DavidAustinNix
Copy link

DavidAustinNix commented Jan 16, 2018 via email

@milos-ljubinkovic
Copy link
Contributor

@DavidAustinNix

Would like to try some steps to see what happens.

  1. Re-run the example with bunny's docker.remove_containers config changed to false. Afterwards you can list the containers with the docker ps -a command and inspect them with the docker inspect {container-id} to see if there's anything in their status/logs that can explain the error.

  2. Uncomment the #executor.override.command=/usr/local/bin/docker run ... config and re-run. This will force rabix to use the command line interface instead of using the tcp socket. (might need to alter the path to the docker client binary if it isn't located in /usr/local/bin)

  3. Try an older version of bunny. Releases 1.0.0-1.0.2 might behave differently as they use a different version of docker client library.

@DavidAustinNix
Copy link

DavidAustinNix commented Jan 16, 2018 via email

@milos-ljubinkovic
Copy link
Contributor

This seems to be the cause:

The paths /Applications/BioApps/Rabix/rabix-cli-1.0.4/examples/dna2protein/data/input.txt and /Applications/BioApps/Rabix/rabix-cli-1.0.4/examples/dna2protein/dna2protein.cwl-2018-01-16-145326.639/root/Transcribe
are not shared from OS X and are not known to Docker.
You can configure shared paths from Docker -> Preferences... -> File Sharing.
See https://docs.docker.com/docker-for-mac/osxfs/#namespaces for more info.

After adding the path (something like /Applications/BioApps/Rabix/rabix-cli-1.0.4/) to the docker shared folders it should work. This is what the window looks like after you click on the docker task bar icon and open the preferences:

image

Weird that this error message wasn't reported on a regular run or that the task didn't fail because of it. I'll see about changing this.

@DavidAustinNix
Copy link

DavidAustinNix commented Jan 16, 2018 via email

@kgaonkar6
Copy link

Hello,

I am getting the same error "Could not find auth config for quay.io. Returning empty builder". I did add the Application folder where I have the RabixComposer and the /Users folder is already shared that has the rabix-cli-1.0.1, following your instructions above in the docker preferences. Can you help identify a solution?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@milos-ljubinkovic @DavidAustinNix @mepstein-gh @kgaonkar6