Skip to content

Commit

Permalink
chore: move host-config and endpoint settings to a specific modifiers (…
Browse files Browse the repository at this point in the history 
…#633)

* feat: support preconfiguring Docker's host config and endpoint settings before container creation

* chore: deprecate ExtraHosts from the container request

* chore: deprecate Binds from the container request

* chore: deprecate Tmpfs from the container request

* chore: deprecate AutoRemove from the container request

* chore: deprecate Privileged from the container request

* chore: deprecate Resources from the container request

* chore: deprecate ShmSize from the container request

* chore: deprecate CapAdd from the container request

* chore: deprecate CapDrop from the container request

* chore: deprecate NetworkMode from the container request

* chore: consistent name for endpointSettings variable

* chore: rename callback to hook

Callback suggests execution after an operation. Using hook following what
git does with commit hooks

* chore: extract default pre-creation hook to a function

* fix: wording

* chore: extract preCreation code to a function

* chore: rename methods to use modifier

* chore: extract life cycle to a separate file

Preparation for the future

* fix: update comments

* chore: push Tmpfs back to layer 1

* chore: push Privileged back to layer 1

* fix: remove outdated comments

* chore: bring ShmSize back to the first layer

* chore: separate concerns for modifiers

* fix: typo in variable

* chore: add modifier for Docker config

In the case of advanced setups, a user would be able to modify anything
in the Docker config

* chore: unit tests for the preCreateHook

* fix: check for the existence of aliases when there are multiple networks

* chore: adjust error messages in tests

* chore: include mounts in the unit tests

* fix: handle error in tests

* fix: handle error in tests

* chore: execute modifiers the last

* chore: rename file

* Revert "chore: execute modifiers the last"

This reverts commit f1b6e26.

* docs: document the modifiers

* fix: typo
  • Loading branch information
@mdelapenya
mdelapenya authored Feb 16, 2023
1 parent a92f198 commit 34481cf
Show file tree
Hide file tree
Showing 8 changed files with 532 additions and 134 deletions.
64 changes: 34 additions & 30 deletions container.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import (

"github.com/docker/docker/api/types"
"github.com/docker/docker/api/types/container"
"github.com/docker/docker/api/types/network"
"github.com/docker/docker/pkg/archive"
"github.com/docker/go-connections/nat"

Expand Down Expand Up @@ -95,36 +96,39 @@ type ContainerFile struct {
// ContainerRequest represents the parameters used to get a running container
type ContainerRequest struct {
FromDockerfile
Image string
Entrypoint []string
Env map[string]string
ExposedPorts []string // allow specifying protocol info
Cmd []string
Labels map[string]string
Mounts ContainerMounts
Tmpfs map[string]string
RegistryCred string
WaitingFor wait.Strategy
Name string // for specifying container name
Hostname string
ExtraHosts []string
Privileged bool // for starting privileged container
Networks []string // for specifying network names
NetworkAliases map[string][]string // for specifying network aliases
NetworkMode container.NetworkMode
Resources container.Resources
Files []ContainerFile // files which will be copied when container starts
User string // for specifying uid:gid
SkipReaper bool // indicates whether we skip setting up a reaper for this
ReaperImage string // Deprecated: use WithImageName ContainerOption instead. Alternative reaper image
ReaperOptions []ContainerOption // options for the reaper
AutoRemove bool // if set to true, the container will be removed from the host when stopped
AlwaysPullImage bool // Always pull image
ImagePlatform string // ImagePlatform describes the platform which the image runs on.
Binds []string
ShmSize int64 // Amount of memory shared with the host (in bytes)
CapAdd []string // Add Linux capabilities
CapDrop []string // Drop Linux capabilities
Image string
Entrypoint []string
Env map[string]string
ExposedPorts []string // allow specifying protocol info
Cmd []string
Labels map[string]string
Mounts ContainerMounts
Tmpfs map[string]string
RegistryCred string
WaitingFor wait.Strategy
Name string // for specifying container name
Hostname string
ExtraHosts []string // Deprecated: Use HostConfigModifier instead
Privileged bool // For starting privileged container
Networks []string // for specifying network names
NetworkAliases map[string][]string // for specifying network aliases
NetworkMode container.NetworkMode // Deprecated: Use HostConfigModifier instead
Resources container.Resources // Deprecated: Use HostConfigModifier instead
Files []ContainerFile // files which will be copied when container starts
User string // for specifying uid:gid
SkipReaper bool // indicates whether we skip setting up a reaper for this
ReaperImage string // Deprecated: use WithImageName ContainerOption instead. Alternative reaper image
ReaperOptions []ContainerOption // options for the reaper
AutoRemove bool // Deprecated: Use HostConfigModifier instead. If set to true, the container will be removed from the host when stopped
AlwaysPullImage bool // Always pull image
ImagePlatform string // ImagePlatform describes the platform which the image runs on.
Binds []string // Deprecated: Use HostConfigModifier instead
ShmSize int64 // Amount of memory shared with the host (in bytes)
CapAdd []string // Deprecated: Use HostConfigModifier instead. Add Linux capabilities
CapDrop []string // Deprecated: Use HostConfigModifier instead. Drop Linux capabilities
ConfigModifier func(*container.Config) // Modifier for the config before container creation
HostConfigModifier func(*container.HostConfig) // Modifier for the host config before container creation
EnpointSettingsModifier func(map[string]*network.EndpointSettings) // Modifier for the network settings before container creation
}

type (
Expand Down
78 changes: 16 additions & 62 deletions docker.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1056,76 +1056,30 @@ func (p *DockerProvider) CreateContainer(ctx context.Context, req ContainerReque
}
}

exposedPorts := req.ExposedPorts
if len(exposedPorts) == 0 && !req.NetworkMode.IsContainer() {
image, _, err := p.client.ImageInspectWithRaw(ctx, tag)
if err != nil {
return nil, err
}
for p := range image.ContainerConfig.ExposedPorts {
exposedPorts = append(exposedPorts, string(p))
}
}

exposedPortSet, exposedPortMap, err := nat.ParsePortSpecs(exposedPorts)
if err != nil {
return nil, err
}

dockerInput := &container.Config{
Entrypoint: req.Entrypoint,
Image: tag,
Env: env,
ExposedPorts: exposedPortSet,
Labels: req.Labels,
Cmd: req.Cmd,
Hostname: req.Hostname,
User: req.User,
Entrypoint: req.Entrypoint,
Image: tag,
Env: env,
Labels: req.Labels,
Cmd: req.Cmd,
Hostname: req.Hostname,
User: req.User,
}

// prepare mounts
mounts := mapToDockerMounts(req.Mounts)

hostConfig := &container.HostConfig{
ExtraHosts: req.ExtraHosts,
PortBindings: exposedPortMap,
Binds: req.Binds,
Mounts: mounts,
Tmpfs: req.Tmpfs,
AutoRemove: req.AutoRemove,
Privileged: req.Privileged,
NetworkMode: req.NetworkMode,
Resources: req.Resources,
ShmSize: req.ShmSize,
CapAdd: req.CapAdd,
CapDrop: req.CapDrop,
}

endpointConfigs := map[string]*network.EndpointSettings{}

// #248: Docker allows only one network to be specified during container creation
// If there is more than one network specified in the request container should be attached to them
// once it is created. We will take a first network if any specified in the request and use it to create container
if len(req.Networks) > 0 {
attachContainerTo := req.Networks[0]

nw, err := p.GetNetwork(ctx, NetworkRequest{
Name: attachContainerTo,
})
if err == nil {
endpointSetting := network.EndpointSettings{
Aliases: req.NetworkAliases[attachContainerTo],
NetworkID: nw.ID,
}
endpointConfigs[attachContainerTo] = &endpointSetting
}
Privileged: req.Privileged,
ShmSize: req.ShmSize,
Tmpfs: req.Tmpfs,
}

networkingConfig := network.NetworkingConfig{
EndpointsConfig: endpointConfigs,
networkingConfig := &network.NetworkingConfig{}

err = p.preCreateContainerHook(ctx, req, dockerInput, hostConfig, networkingConfig)
if err != nil {
return nil, err
}

resp, err := p.client.ContainerCreate(ctx, dockerInput, hostConfig, &networkingConfig, platform, req.Name)
resp, err := p.client.ContainerCreate(ctx, dockerInput, hostConfig, networkingConfig, platform, req.Name)
if err != nil {
return nil, err
}
Expand Down
74 changes: 45 additions & 29 deletions docker_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -141,15 +141,17 @@ func TestContainerWithHostNetworkOptions(t *testing.T) {
gcr := GenericContainerRequest{
ProviderType: providerType,
ContainerRequest: ContainerRequest{
Image: nginxAlpineImage,
Privileged: true,
SkipReaper: true,
NetworkMode: "host",
Mounts: Mounts(BindMount(absPath, "/etc/nginx/conf.d/default.conf")),
Image: nginxAlpineImage,
SkipReaper: true,
Mounts: Mounts(BindMount(absPath, "/etc/nginx/conf.d/default.conf")),
ExposedPorts: []string{
nginxHighPort,
},
Privileged: true,
WaitingFor: wait.ForListeningPort(nginxHighPort),
HostConfigModifier: func(hc *container.HostConfig) {
hc.NetworkMode = "host"
},
},
Started: true,
}
Expand Down Expand Up @@ -209,10 +211,12 @@ func TestContainerWithNetworkModeAndNetworkTogether(t *testing.T) {
gcr := GenericContainerRequest{
ProviderType: providerType,
ContainerRequest: ContainerRequest{
Image: nginxImage,
SkipReaper: true,
NetworkMode: "host",
Networks: []string{"new-network"},
Image: nginxImage,
SkipReaper: true,
Networks: []string{"new-network"},
HostConfigModifier: func(hc *container.HostConfig) {
hc.NetworkMode = "host"
},
},
Started: true,
}
Expand All @@ -236,11 +240,13 @@ func TestContainerWithHostNetworkOptionsAndWaitStrategy(t *testing.T) {
gcr := GenericContainerRequest{
ProviderType: providerType,
ContainerRequest: ContainerRequest{
Image: nginxAlpineImage,
SkipReaper: true,
NetworkMode: "host",
WaitingFor: wait.ForListeningPort(nginxHighPort),
Mounts: Mounts(BindMount(absPath, "/etc/nginx/conf.d/default.conf")),
Image: nginxAlpineImage,
SkipReaper: true,
WaitingFor: wait.ForListeningPort(nginxHighPort),
Mounts: Mounts(BindMount(absPath, "/etc/nginx/conf.d/default.conf")),
HostConfigModifier: func(hc *container.HostConfig) {
hc.NetworkMode = "host"
},
},
Started: true,
}
Expand Down Expand Up @@ -272,11 +278,13 @@ func TestContainerWithHostNetworkAndEndpoint(t *testing.T) {
gcr := GenericContainerRequest{
ProviderType: providerType,
ContainerRequest: ContainerRequest{
Image: nginxAlpineImage,
SkipReaper: true,
NetworkMode: "host",
WaitingFor: wait.ForListeningPort(nginxHighPort),
Mounts: Mounts(BindMount(absPath, "/etc/nginx/conf.d/default.conf")),
Image: nginxAlpineImage,
SkipReaper: true,
WaitingFor: wait.ForListeningPort(nginxHighPort),
Mounts: Mounts(BindMount(absPath, "/etc/nginx/conf.d/default.conf")),
HostConfigModifier: func(hc *container.HostConfig) {
hc.NetworkMode = "host"
},
},
Started: true,
}
Expand Down Expand Up @@ -309,11 +317,13 @@ func TestContainerWithHostNetworkAndPortEndpoint(t *testing.T) {
gcr := GenericContainerRequest{
ProviderType: providerType,
ContainerRequest: ContainerRequest{
Image: nginxAlpineImage,
SkipReaper: true,
NetworkMode: "host",
WaitingFor: wait.ForListeningPort(nginxHighPort),
Mounts: Mounts(BindMount(absPath, "/etc/nginx/conf.d/default.conf")),
Image: nginxAlpineImage,
SkipReaper: true,
WaitingFor: wait.ForListeningPort(nginxHighPort),
Mounts: Mounts(BindMount(absPath, "/etc/nginx/conf.d/default.conf")),
HostConfigModifier: func(hc *container.HostConfig) {
hc.NetworkMode = "host"
},
},
Started: true,
}
Expand Down Expand Up @@ -2317,8 +2327,10 @@ func TestDockerContainerResources(t *testing.T) {
Image: nginxAlpineImage,
ExposedPorts: []string{nginxDefaultPort},
WaitingFor: wait.ForListeningPort(nginxDefaultPort),
Resources: container.Resources{
Ulimits: expected,
HostConfigModifier: func(hc *container.HostConfig) {
hc.Resources = container.Resources{
Ulimits: expected,
}
},
},
Started: true,
Expand Down Expand Up @@ -2403,7 +2415,9 @@ func TestContainerCapAdd(t *testing.T) {
Image: nginxAlpineImage,
ExposedPorts: []string{nginxDefaultPort},
WaitingFor: wait.ForListeningPort(nginxDefaultPort),
CapAdd: []string{expected},
HostConfigModifier: func(hc *container.HostConfig) {
hc.CapAdd = []string{expected}
},
},
Started: true,
})
Expand Down Expand Up @@ -2547,8 +2561,10 @@ func TestNetworkModeWithContainerReference(t *testing.T) {
nginxB, err := GenericContainer(ctx, GenericContainerRequest{
ProviderType: providerType,
ContainerRequest: ContainerRequest{
Image: nginxAlpineImage,
NetworkMode: container.NetworkMode(networkMode),
Image: nginxAlpineImage,
HostConfigModifier: func(hc *container.HostConfig) {
hc.NetworkMode = container.NetworkMode(networkMode)
},
},
Started: true,
})
Expand Down
11 changes: 11 additions & 0 deletions docs/features/creating_container.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,17 @@ func TestIntegrationNginxLatestReturn(t *testing.T) {
}
```

### Advanced Settings

The aforementioned `GenericContainer` function and the `ContainerRequest` struct represent a straightforward manner to configure the containers, but you could need to create your containers with more advance settings regarding the config, host config and endpoint settings Docker types. For those more advance settings, _Testcontainers for Go_ offers a way to fully customise the container request and those internal Docker types. These customisations, called _modifiers_, will be applied just before the internal call to the Docker client to create the container.

<!--codeinclude-->
[Using modifiers](../../lifecycle_test.go) inside_block:reqWithModifiers
<!--/codeinclude-->

!!!warning
The only special case where the modifiers are not applied last, is when there are no exposed ports in the container request and the container does not use a network mode from a container (e.g. `req.NetworkMode = container.NetworkMode("container:$CONTAINER_ID")`). In that case, _Testcontainers for Go_ will extract the ports from the underliying Docker image and export them.

## Reusable container

With `Reuse` option you can reuse an existing container. Reusing will work only if you pass an
Expand Down
Loading

0 comments on commit 34481cf

Please sign in to comment.