Docker authorization failed on kamal deploy

[csalmeida]

Hi everyone, I have been managing to deploy an app with Kamal with these Docker credentials for a few days with no issues but now I am getting the following error and I'm unsure as to why that might be happening.

The image is available in the registry and just to see if the token was bad I created a new one, added it to .env and ran kamal env push before trying again but I still got the same error just like below, has anyone experienced this one?

------
 > exporting to image:
------
ERROR: failed to solve: failed to push dockerusername/service-name:4z484256423y7c90x2409efb385aa447df33b2fa: push access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Note

I run Kamal using the following alias (not the gem install):
alias kamal='docker run -it --rm -v "${PWD}:/workdir" -v "${HOME}/.ssh:/root/.ssh" -v "/run/host-services/ssh-auth.sock:/run/host-services/ssh-auth.sock" -e SSH_AUTH_SOCK="/run/host-services/ssh-auth.sock" -v /var/run/docker.sock:/var/run/docker.sock ghcr.io/basecamp/kamal:latest'

[Briscoooe]

I am also getting this. My config section for this is

# Credentials for your image host.
registry:
  username: myusername

  # Always use an access token rather than real password when possible.
  password:
    - KAMAL_REGISTRY_PASSWORD

where KAMAL_REGISTRY_PASSWORD is in my .env file. It is a Dockerhub access token with the following permissions.

What am I missing?

Here's the output of kamal registry login

  INFO [e99f5161] Running docker login -u [REDACTED] -p [REDACTED] on localhost
  INFO [e99f5161] Finished in 1.498 seconds with exit status 0 (successful).
  INFO [a933d128] Running docker login -u [REDACTED] -p [REDACTED] on ec2-[...].eu-west-1.compute.amazonaws.com
  INFO [a933d128] Finished in 1.396 seconds with exit status 0 (successful).

[viktorianer]

Solved for me too.

But, this sounds like a bug? Should be solved or is there some limitation?

[tillcarlos]

Oh, what a relief! This just took me hours to debug. Thanks @r3cha !

[tillcarlos]

Now I also have this problem on my CI (gitlab selfhosted + external runner, amd64).

• First logins are okay
• it fails when pushing the image
• locally it works, my token has "read_repository, read_registry, write_registry, read_package_registry, write_package_registry"
• there is no builder to remove: no builder "kamal-local-docker-container" found

I guess it's the same issue. Any idea how to get rid of the credentials cache?

[fesilvajacquier]

Thank you @r3cha 🙌
After changing Docker Hub accounts I started encountering this error. Rming the containers solved it.

[vickodin]

@r3cha Thank you, bro!

[yaKashif]

Has anyone solved this?

[jlenriquez]

Hi @yaKashif can you check the name of the image container (prior the forward slash / ) and registry username if it's the same? If the image container name is not same with the registry name this might be the possible cause of the issue.

[yaKashif]

It is same.

[yaKashif]

adding server: registry.hub.docker.com in deploy.yml worked

I though it would default to that.

[jlenriquez]

On my end I didn’t need to add the server as I also have the same understanding that it would be the default.

…

On Sat, Oct 19, 2024 at 14:41 Kashif ***@***.***> wrote: adding server: registry.hub.docker.com in deploy.yml worked I though it would default to that. — Reply to this email directly, view it on GitHub <#896 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAE7TRSJV5CWASUONDAZGKDZ4HWHXAVCNFSM6AAAAABLSGM22GVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAOJYHEZTSOI> . You are receiving this because you commented.Message ID: ***@***.***>

[stefatkins]

Hi everyone,

I’m encountering an issue with kamal and my GitHub Actions workflow during deployment. On my local environment, both kamal deploy and kamal redeploy work without any issues. However, when I use kamal redeploy in the GitHub Actions workflow, the deployment fails. Strangely, if I switch it to kamal deploy in the workflow, it works perfectly. I'm using a Read, Write, Delete docker acess token.

error is : push access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

Has anyone else faced this problem or have any insights on why kamal redeploy might be failing in the GitHub Actions environment but works fine locally?

name: Deploy
on:
  workflow_run:
    workflows: ["CI"]
    types:
      - completed
jobs:
  deploy:
    if: ${{ github.event.workflow_run.conclusion == 'success' }}
    runs-on: ubuntu-latest
    container:
      image: ghcr.io/basecamp/kamal:latest
    steps:
      - name: Checkout repository
        uses: actions/checkout@v4
      - name: Set up SSH
        run: |
          mkdir -p ~/.ssh
          echo "${{ secrets.SSH_PRIVATE_KEY }}" > ~/.ssh/id_rsa
          chmod 600 ~/.ssh/id_rsa
      - name: Disable Git safe directory check
        run: git config --global --add safe.directory '*'
      - name: Deploy Application
        env:
          KAMAL_REGISTRY_PASSWORD: ${{ secrets.KAMAL_REGISTRY_PASSWORD }}
          RAILS_MASTER_KEY: ${{ secrets.RAILS_MASTER_KEY }}
        run: |
          kamal lock release && kamal redeploy

[tannakartikey]

I faced that issues with GitHub Action. After few hours, it started working again. I thought it must have hit rate limit of Docker Hub. In that case I should have received 429 instead of 401 though. Not sure what went wrong.

Meanwhile, it was working fine when I triggered the flow from my laptop.

[stefatkins]

I faced that issues with GitHub Action. After few hours, it started working again. I thought it must have hit rate limit of Docker Hub. In that case I should have received 429 instead of 401 though. Not sure what went wrong.

Meanwhile, it was working fine when I triggered the flow from my laptop.

Strange because I have been trying all morning and it is still not working.

[tannakartikey]

I think the issue is that "redeploy" doesn't login into the registry. It's mentioned in the docs.

[stefatkins]

Ah I must of missed that part. Thanks Tanna

[jgmontoya]

I had this issue and solved it by doing docker login

[csalmeida]

In my case, it simply started working again with no explanation. I am unsure at this point what the cause might have been but I can close it and someone else can re-open if they have a similar issue.