Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: support reading Docker host from the current Docker context #2810

Open
wants to merge 56 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
6c1b3d8
feat: support reading Docker context
mdelapenya Oct 2, 2024
415c896
chore: move read Docker config to the core
mdelapenya Oct 2, 2024
1e620f1
chore: read docker config more consistently
mdelapenya Oct 2, 2024
ea44c92
chore: extract to a parse function to detect remote docker hosts
mdelapenya Oct 2, 2024
549b1cb
docs: add Docker context to docs
mdelapenya Oct 2, 2024
fbefbdf
fix: lint
mdelapenya Oct 2, 2024
1ca3f63
chore: simplify container provider initialisation
mdelapenya Oct 3, 2024
7e1e4b5
feat: support configuring the bridge network name
mdelapenya Oct 3, 2024
2decbe3
chore: rename function
mdelapenya Oct 3, 2024
c596e37
fix: remove wrong copy&paste
mdelapenya Oct 3, 2024
ece6ea5
chore: simplify
mdelapenya Oct 3, 2024
a775c4b
chore: simplify error
mdelapenya Oct 3, 2024
5ac2263
chore: simple naming in test
mdelapenya Oct 3, 2024
932fce2
fix: do not lose the original error
mdelapenya Oct 3, 2024
9c048f7
fix: line endings
mdelapenya Oct 3, 2024
77cdda6
fix: case insensitive regex
mdelapenya Oct 4, 2024
7eab89f
chore: deprecate reaper_network and always use the bridge one
mdelapenya Oct 4, 2024
1e58328
chore: support replacing the bridge network in the endpoint modifier
mdelapenya Oct 4, 2024
8abfeba
fix: lint
mdelapenya Oct 4, 2024
203d762
chore: assume the container runtime uses the default network by default
mdelapenya Oct 4, 2024
464bc9b
chore: rename variable to avoid shading package
mdelapenya Oct 5, 2024
37a8142
fix: if the properties file does not exist, use default config
mdelapenya Oct 5, 2024
07003ff
fix: typo
mdelapenya Oct 5, 2024
adf0ba1
fix: use default config in tests
mdelapenya Oct 7, 2024
b31da78
docs: simplify
mdelapenya Oct 14, 2024
a38a345
chore: simplify error when docker context is not found
mdelapenya Oct 14, 2024
14aa3dd
fix: check if the docker socket is listening
mdelapenya Oct 16, 2024
b327978
docs: document the docker context support
mdelapenya Oct 16, 2024
33528c6
chore: remove bridge network custom configuration
mdelapenya Oct 16, 2024
ca07e04
Merge branch 'main' into context-support
mdelapenya Oct 18, 2024
dab5614
Merge branch 'main' into context-support
mdelapenya Oct 21, 2024
a313dc5
chore: use t.Helper
mdelapenya Oct 21, 2024
5511d17
Merge branch 'main' into context-support
mdelapenya Oct 28, 2024
b38e857
Merge branch 'main' into context-support
mdelapenya Nov 19, 2024
bc2f04b
chore: proper deprecation path
mdelapenya Nov 19, 2024
dc571d3
Merge branch 'main' into context-support
mdelapenya Nov 25, 2024
ad79b43
chore: comment out empty field used for documentation
mdelapenya Nov 25, 2024
4d4769f
chore: use require
mdelapenya Nov 25, 2024
56d6774
docs: update podman commands
mdelapenya Nov 25, 2024
1273e02
chore: readability in var comments
mdelapenya Nov 25, 2024
e7dd83c
chore: unique parsing error messages
mdelapenya Nov 25, 2024
584f6da
docs: readability as function comments
mdelapenya Nov 25, 2024
3b59086
chore: idiomatic var declaration
mdelapenya Nov 25, 2024
1c93fe8
chore: extract file name to constant
mdelapenya Nov 25, 2024
92d306f
chore: simplify default config
mdelapenya Nov 25, 2024
499a05d
chore: make errors private
mdelapenya Nov 25, 2024
cdb874d
chore: simplify function
mdelapenya Nov 25, 2024
1933f42
fix: remove unused func
mdelapenya Nov 25, 2024
f4783a7
chore: bubble up config errors when when config is incorrect
mdelapenya Dec 4, 2024
28f0802
chore: use better test names
mdelapenya Dec 4, 2024
f8d24c7
chore: deprecate ReadConfig in favour of NewConfig
mdelapenya Dec 5, 2024
786d79c
chore: line separator
mdelapenya Dec 5, 2024
4bd9934
chore: do not export var
mdelapenya Dec 5, 2024
5f7eb43
Merge branch 'main' into context-support
mdelapenya Dec 5, 2024
8fcd654
chore: bubble up error on environment configuration
mdelapenya Dec 5, 2024
13934da
chore: rename test cases
mdelapenya Dec 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 1 addition & 23 deletions docker_auth.go
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ func configKey(cfg *dockercfg.Config) (string, error) {
// getDockerAuthConfigs returns a map with the auth configs from the docker config file
// using the registry as the key
func getDockerAuthConfigs() (map[string]registry.AuthConfig, error) {
cfg, err := getDockerConfig()
cfg, err := core.ReadDockerConfig()
if err != nil {
if errors.Is(err, os.ErrNotExist) {
return map[string]registry.AuthConfig{}, nil
Expand Down Expand Up @@ -246,25 +246,3 @@ func getDockerAuthConfigs() (map[string]registry.AuthConfig, error) {

return cfgs, nil
}

// getDockerConfig returns the docker config file. It will internally check, in this particular order:
// 1. the DOCKER_AUTH_CONFIG environment variable, unmarshalling it into a dockercfg.Config
// 2. the DOCKER_CONFIG environment variable, as the path to the config file
// 3. else it will load the default config file, which is ~/.docker/config.json
func getDockerConfig() (*dockercfg.Config, error) {
if env := os.Getenv("DOCKER_AUTH_CONFIG"); env != "" {
var cfg dockercfg.Config
if err := json.Unmarshal([]byte(env), &cfg); err != nil {
return nil, fmt.Errorf("unmarshal DOCKER_AUTH_CONFIG: %w", err)
}

return &cfg, nil
}

cfg, err := dockercfg.LoadDefaultConfig()
if err != nil {
return nil, fmt.Errorf("load default config: %w", err)
}

return &cfg, nil
}
80 changes: 0 additions & 80 deletions docker_auth_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,82 +16,11 @@ import (
"github.com/docker/docker/client"
"github.com/stretchr/testify/require"

"github.com/testcontainers/testcontainers-go/internal/core"
"github.com/testcontainers/testcontainers-go/wait"
)

const exampleAuth = "https://example-auth.com"

func Test_getDockerConfig(t *testing.T) {
expectedConfig := &dockercfg.Config{
AuthConfigs: map[string]dockercfg.AuthConfig{
core.IndexDockerIO: {},
"https://example.com": {},
"https://my.private.registry": {},
},
CredentialsStore: "desktop",
}
t.Run("HOME/valid", func(t *testing.T) {
testDockerConfigHome(t, "testdata")

cfg, err := getDockerConfig()
require.NoError(t, err)
require.Equal(t, expectedConfig, cfg)
})

t.Run("HOME/not-found", func(t *testing.T) {
testDockerConfigHome(t, "testdata", "not-found")

cfg, err := getDockerConfig()
require.ErrorIs(t, err, os.ErrNotExist)
require.Nil(t, cfg)
})

t.Run("HOME/invalid-config", func(t *testing.T) {
testDockerConfigHome(t, "testdata", "invalid-config")

cfg, err := getDockerConfig()
require.ErrorContains(t, err, "json: cannot unmarshal array")
require.Nil(t, cfg)
})

t.Run("DOCKER_AUTH_CONFIG/valid", func(t *testing.T) {
testDockerConfigHome(t, "testdata", "not-found")
t.Setenv("DOCKER_AUTH_CONFIG", dockerConfig)

cfg, err := getDockerConfig()
require.NoError(t, err)
require.Equal(t, expectedConfig, cfg)
})

t.Run("DOCKER_AUTH_CONFIG/invalid-config", func(t *testing.T) {
testDockerConfigHome(t, "testdata", "not-found")
t.Setenv("DOCKER_AUTH_CONFIG", `{"auths": []}`)

cfg, err := getDockerConfig()
require.ErrorContains(t, err, "json: cannot unmarshal array")
require.Nil(t, cfg)
})

t.Run("DOCKER_CONFIG/valid", func(t *testing.T) {
testDockerConfigHome(t, "testdata", "not-found")
t.Setenv("DOCKER_CONFIG", filepath.Join("testdata", ".docker"))

cfg, err := getDockerConfig()
require.NoError(t, err)
require.Equal(t, expectedConfig, cfg)
})

t.Run("DOCKER_CONFIG/invalid-config", func(t *testing.T) {
testDockerConfigHome(t, "testdata", "not-found")
t.Setenv("DOCKER_CONFIG", filepath.Join("testdata", "invalid-config", ".docker"))

cfg, err := getDockerConfig()
require.ErrorContains(t, err, "json: cannot unmarshal array")
require.Nil(t, cfg)
})
}

func TestDockerImageAuth(t *testing.T) {
t.Run("retrieve auth with DOCKER_AUTH_CONFIG env var", func(t *testing.T) {
username, password := "gopher", "secret"
Expand Down Expand Up @@ -421,15 +350,6 @@ func Test_getDockerAuthConfigs(t *testing.T) {

requireValidAuthConfig(t)
})

t.Run("DOCKER_CONFIG/invalid-config", func(t *testing.T) {
testDockerConfigHome(t, "testdata", "not-found")
t.Setenv("DOCKER_CONFIG", filepath.Join("testdata", "invalid-config", ".docker"))

cfg, err := getDockerConfig()
require.ErrorContains(t, err, "json: cannot unmarshal array")
require.Nil(t, cfg)
})
}

// requireValidAuthConfig checks that the given authConfigs map contains the expected keys.
Expand Down
10 changes: 6 additions & 4 deletions docs/features/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,17 +75,19 @@ See [Docker environment variables](https://docs.docker.com/engine/reference/comm

3. Read the Go context for the **DOCKER_HOST** key. E.g. `ctx.Value("DOCKER_HOST")`. This is used internally for the library to pass the Docker host to the resource reaper.

4. Read the default Docker socket path, without the unix schema. E.g. `/var/run/docker.sock`
4. Read the host endpoint for the current Docker context in the Docker configuration file. E.g. `~/.docker/config.json`.
mdelapenya marked this conversation as resolved.
Show resolved Hide resolved

5. Read the **docker.host** property in the `~/.testcontainers.properties` file. E.g. `docker.host=tcp://my.docker.host:1234`
5. Read the default Docker socket path, without the unix schema. E.g. `/var/run/docker.sock`
mdelapenya marked this conversation as resolved.
Show resolved Hide resolved

6. Read the rootless Docker socket path, checking in the following alternative locations:
6. Read the **docker.host** property in the `~/.testcontainers.properties` file. E.g. `docker.host=tcp://my.docker.host:1234`

7. Read the rootless Docker socket path, checking in the following alternative locations:
1. `${XDG_RUNTIME_DIR}/.docker/run/docker.sock`.
2. `${HOME}/.docker/run/docker.sock`.
3. `${HOME}/.docker/desktop/docker.sock`.
4. `/run/user/${UID}/docker.sock`, where `${UID}` is the user ID of the current user.

7. The library panics if none of the above are set, meaning that the Docker host was not detected.
8. The library panics if none of the above are set, meaning that the Docker host was not detected.

## Docker socket path detection

Expand Down
2 changes: 1 addition & 1 deletion generic.go
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ func GenericContainer(ctx context.Context, req GenericContainerRequest) (Contain
// TODO: Remove this debugging.
if strings.Contains(err.Error(), "toomanyrequests") {
// Debugging information for rate limiting.
cfg, err := getDockerConfig()
cfg, err := core.ReadDockerConfig()
if err == nil {
fmt.Printf("XXX: too many requests: %+v", cfg)
}
Expand Down
31 changes: 31 additions & 0 deletions internal/core/docker_config.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
package core

import (
"encoding/json"
"fmt"
"os"

"github.com/cpuguy83/dockercfg"
)

// ReadDockerConfig returns the docker config file. It will internally check, in this particular order:
// 1. the DOCKER_AUTH_CONFIG environment variable, unmarshalling it into a dockercfg.Config
// 2. the DOCKER_CONFIG environment variable, as the path to the config file
// 3. else it will load the default config file, which is ~/.docker/config.json
func ReadDockerConfig() (*dockercfg.Config, error) {
mdelapenya marked this conversation as resolved.
Show resolved Hide resolved
if env := os.Getenv("DOCKER_AUTH_CONFIG"); env != "" {
var cfg dockercfg.Config
if err := json.Unmarshal([]byte(env), &cfg); err != nil {
return nil, fmt.Errorf("unmarshal DOCKER_AUTH_CONFIG: %w", err)
}

return &cfg, nil
}

cfg, err := dockercfg.LoadDefaultConfig()
if err != nil {
return nil, fmt.Errorf("load default config: %w", err)
}

return &cfg, nil
}
96 changes: 96 additions & 0 deletions internal/core/docker_config_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
package core

import (
_ "embed"
"os"
"path/filepath"
"testing"

"github.com/cpuguy83/dockercfg"
"github.com/stretchr/testify/require"
)

//go:embed testdata/.docker/config.json
var dockerConfig string

func TestReadDockerConfig(t *testing.T) {
expectedConfig := &dockercfg.Config{
AuthConfigs: map[string]dockercfg.AuthConfig{
IndexDockerIO: {},
"https://example.com": {},
"https://my.private.registry": {},
},
CredentialsStore: "desktop",
}
t.Run("HOME/valid", func(t *testing.T) {
testDockerConfigHome(t, "testdata")

cfg, err := ReadDockerConfig()
require.NoError(t, err)
require.Equal(t, expectedConfig, cfg)
})

t.Run("HOME/not-found", func(t *testing.T) {
testDockerConfigHome(t, "testdata", "not-found")

cfg, err := ReadDockerConfig()
require.ErrorIs(t, err, os.ErrNotExist)
require.Nil(t, cfg)
})

t.Run("HOME/invalid-config", func(t *testing.T) {
testDockerConfigHome(t, "testdata", "invalid-config")

cfg, err := ReadDockerConfig()
require.ErrorContains(t, err, "json: cannot unmarshal array")
require.Nil(t, cfg)
})

t.Run("DOCKER_AUTH_CONFIG/valid", func(t *testing.T) {
testDockerConfigHome(t, "testdata", "not-found")
t.Setenv("DOCKER_AUTH_CONFIG", dockerConfig)

cfg, err := ReadDockerConfig()
require.NoError(t, err)
require.Equal(t, expectedConfig, cfg)
})

t.Run("DOCKER_AUTH_CONFIG/invalid-config", func(t *testing.T) {
testDockerConfigHome(t, "testdata", "not-found")
t.Setenv("DOCKER_AUTH_CONFIG", `{"auths": []}`)

cfg, err := ReadDockerConfig()
require.ErrorContains(t, err, "json: cannot unmarshal array")
require.Nil(t, cfg)
})

t.Run("DOCKER_CONFIG/valid", func(t *testing.T) {
testDockerConfigHome(t, "testdata", "not-found")
t.Setenv("DOCKER_CONFIG", filepath.Join("testdata", ".docker"))

cfg, err := ReadDockerConfig()
require.NoError(t, err)
require.Equal(t, expectedConfig, cfg)
})

t.Run("DOCKER_CONFIG/invalid-config", func(t *testing.T) {
testDockerConfigHome(t, "testdata", "not-found")
t.Setenv("DOCKER_CONFIG", filepath.Join("testdata", "invalid-config", ".docker"))

cfg, err := ReadDockerConfig()
require.ErrorContains(t, err, "json: cannot unmarshal array")
require.Nil(t, cfg)
})
}

// testDockerConfigHome sets the user's home directory to the given path
// and unsets the DOCKER_CONFIG and DOCKER_AUTH_CONFIG environment variables.
func testDockerConfigHome(t *testing.T, dirs ...string) {
t.Helper()

dir := filepath.Join(dirs...)
t.Setenv("DOCKER_AUTH_CONFIG", "")
t.Setenv("DOCKER_CONFIG", "")
t.Setenv("HOME", dir)
t.Setenv("USERPROFILE", dir) // Windows
}
Loading