Unable to have tests that use testcontainers running successful in a docker image

[joaocpribeiro]

In older projects I have been running our tests in a docker image in devops build pipeline.
In my new project I am trying to use testcontainers and I would like to follow the same approach. However I am facing an issue.

While investigating, I found this discussion, which might have similarities, but it did not help me so far. Taking that discussion, I decided to use @HofmeisterAn example (code here).

When I run these tests locally with dotnet test -c Release --results-directory /testresults --logger trx --logger=console;verbosity=detailed "Issue.Tests.csproj", the tests run successfully and I get the output Standard Output Messages: localhost.

If I add the following dockerfile, I get an error while building it.

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ["Issue.Tests.csproj", "."]
RUN dotnet restore "./Issue.Tests.csproj"
COPY . .

FROM build AS tests
WORKDIR "/src/."
RUN dotnet test -c Release --results-directory /testresults --logger trx --logger=console;verbosity=detailed "Issue.Tests.csproj"

Call: docker build --progress=plain --target tests --tag tc679 . (on local machine)

Error:

#11 [tests 2/2] RUN dotnet test -c Release --results-directory /testresults --logger trx --logger=console;verbosity=detailed "Issue.Tests.csproj"
#11 sha256:954cb744afe575447c3e5a81b831d7e42ecf2ef917fe550d86101cb4c3f170a8
#11 3.755   Determining projects to restore...
#11 4.884   All projects are up-to-date for restore.
#11 12.07   Issue.Tests -> /src/bin/Release/net6.0/Issue.Tests.dll
#11 12.10 Test run for /src/bin/Release/net6.0/Issue.Tests.dll (.NETCoreApp,Version=v6.0)
#11 12.36 Microsoft (R) Test Execution Command Line Tool Version 17.3.1 (x64)
#11 12.36 Copyright (c) Microsoft Corporation.  All rights reserved.
#11 12.37
#11 12.65 Starting test execution, please wait...
#11 12.80 A total of 1 test files matched the specified pattern.
#11 16.53 [xUnit.net 00:00:02.17]     GitHub.GetHostname [FAIL]
#11 16.77   Failed GitHub.GetHostname [1 ms]
#11 16.77   Error Message:
#11 16.77    System.ArgumentNullException : Cannot detect the Docker endpoint. Use either the environment variables or the ~/.testcontainers.properties file to customize your configuration:
#11 16.77 https://www.testcontainers.org/features/configuration/#customizing-docker-host-detection (Parameter 'DockerEndpointAuthConfig')
#11 16.77   Stack Trace:
#11 16.77      at DotNet.Testcontainers.Guard.DockerEndpointAuthConfigIsSet[TType](ArgumentInfo`1& argument)
#11 16.77    at DotNet.Testcontainers.Builders.TestcontainersBuilder`1.Build()
#11 16.77    at GitHub..ctor(ITestOutputHelper testOutputHelper) in /src/GitHub.cs:line 9
#11 16.81 Results File: /testresults/_buildkitsandbox_2023-01-11_16_26_37.trx
#11 16.81
#11 16.82 Failed!  - Failed:     1, Passed:     0, Skipped:     0, Total:     1, Duration: < 1 ms - /src/bin/Release/net6.0/Issue.Tests.dll (net6.0)

My knowledge in docker is not deep. I understand the the tests within the docker image do not know how to communicate with docker to manage the testcontainers, but I do not know what I should configure as DockerEndpointAuthConfig or other docker variables.

Could someone please guide me?
Thank you in advance.

[HofmeisterAn]

I understand the the tests within the docker image do not know how to communicate with docker

Kind of, yes. Your Docker image does not include Docker. You need to expose a Docker endpoint. Either via TCP or something like Docker-in-Docker (DinD). #492 (comment) explains two different DinD approaches. Maybe the Docker Desktop Compose example helps to understand the pattern for sibling containers too. @kiview do you have a more detailed example? Although, I am not sure how to do it during docker build. To run Testcontainers in the docker build process see my answer at Stack Overflow

[joaocpribeiro]

Thank you, @HofmeisterAn! I went on with the sibling approach and docker compose. Also I am not executing the tests in the build anymore. It might be okay without external dependencies (outside the container -> using testcontainers), but with these dependencies it makes more sense to run them afterwards.
I was able to run the tests successfully in a container on my machine. I am now trying out in DevOps. The tests fail there, due to DotNet.Testcontainers.Containers.ResourceReaperException : Initialization has been cancelled. I know this was an issue before in other cases, and I also experienced it in my machine before due to another docker container with similar tests and dependencies. I will investigate further on my own, according to my availability.
I can share the changed code afterwards, if I struggle again or when I fix all my issues. :)

[HofmeisterAn]

Getting ResourceReaperException with the latest version is unusual. Does it happen frequently? If you like, you can ping me on Slack and we can take a look at your configuration.

[joaocpribeiro]

It happens every time in DevOps.

[HofmeisterAn]

Ok, that sounds like a misconfiguration. Here is a simple example that runs on the MS-hosted agent. This one runs inside a container. Maybe one of them help.

[joaocpribeiro]

As we discussed in Slack, yes, it was a misconfiguration. Here is my current solution, for anyone interested:

My Testcontainer:

        private readonly TestcontainersContainer _dbContainer =
          new TestcontainersBuilder<TestcontainersContainer>()
            .WithImage("mcr.microsoft.com/mssql/server:2022-latest")
            .WithPortBinding(MssqlContainerPort, true)
            .WithEnvironment("ACCEPT_EULA", "Y")
            .WithEnvironment("MSSQL_SA_PASSWORD", Password)
            .WithWaitStrategy(Wait.ForUnixContainer().UntilCommandIsCompleted("/opt/mssql-tools/bin/sqlcmd", "-S", $"localhost,{MssqlContainerPort}", "-U", Username, "-P", Password))
            .Build();

Dockefile:

FROM mcr.microsoft.com/dotnet/sdk:6.0 AS build
WORKDIR /src
COPY ["Issue.Tests/Issue.Tests.csproj", "Issue.Tests/"]
RUN dotnet restore "Issue.Tests/Issue.Tests.csproj"
COPY . .

docker-compose.yml:

version: "3"

services:
  issue.tests:
    build:
      context: .
      dockerfile: Issue.Tests/Dockerfile
    entrypoint: dotnet
    command: test Issue.Tests/Issue.Tests.csproj
    #environment:
    #  - TESTCONTAINERS_HOST_OVERRIDE=host.docker.internal -> needed if running in windows machine, with docker desktop
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock

And I am running the tests in devops pipeline with docker compose -f docker-compose.yml up issue.tests.

If running in windows machine, with docker desktop, an override yml file is needed (docker-compose.debug.yml in this case):

version: "3"

services:
  issue.tests:
    environment:
      - TESTCONTAINERS_HOST_OVERRIDE=host.docker.internal

and running docker compose -f docker-compose.yml -f docker-compose.debug.yml up issue.tests.