Unsure if there is a bug when using ".WithPortBinding()" or if I'm not using TestContainers correctly

[stefanteodoropol]

Hello Everyone!

I am trying to Use TestContainers in a DotNet project, specifically to run a PostgresSql instance for some Test.

In order to simplify debugging I am trying to fix the selected port of the container using .WithPortBinding("3033", "5432")

I get an error however after the first test runs. The second test class throws an exception. Then the third test works, The forth throws an exception, and so on.

` Message: 
Npgsql.NpgsqlException : Exception while reading from stream
----> System.IO.IOException : Unable to read data from the transport connection: An established connection was aborted by the software in your host machine..
----> System.Net.Sockets.SocketException : An established connection was aborted by the software in your host machine.

Stack Trace: 
NpgsqlReadBuffer.g__EnsureLong|55_0(NpgsqlReadBuffer buffer, Int32 count, Boolean async, Boolean readingNotifications)
IValueTaskSource.GetResult(Int16 token)
NpgsqlConnector.ReadMessageLong(Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)
IValueTaskSource.GetResult(Int16 token)
NpgsqlConnector.ReadMessageLong(Boolean async, DataRowLoadingMode dataRowLoadingMode, Boolean readingNotifications, Boolean isReadingPrependedMessage)
IValueTaskSource.GetResult(Int16 token)
NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
NpgsqlDataReader.NextResult(Boolean async, Boolean isConsuming, CancellationToken cancellationToken)
NpgsqlCommand.ExecuteReader(Boolean async, CommandBehavior behavior, CancellationToken cancellationToken)
NpgsqlCommand.ExecuteReader(Boolean async, CommandBehavior behavior, CancellationToken cancellationToken)
<67 more frames...>
WorkItem.RunOnCurrentThread()
WorkItem.Execute()
ParallelWorkItemDispatcher.Dispatch(WorkItem work, ParallelExecutionStrategy strategy)
ParallelWorkItemDispatcher.Dispatch(WorkItem work)
CompositeWorkItem.RunChildren()
CompositeWorkItem.PerformWork()
WorkItem.RunOnCurrentThread()
WorkItem.Execute()
TestWorker.TestWorkerThreadProc()
--- End of stack trace from previous location ---`

I'm not sure if this is a bug or if I am doing something wrong? I tried to create a minimal example to showcase the behavior here:

https://github.com/stefanteodoropol/Testcontainer_Port_Bug_Demo/

I appreciate any help in advance regarding this issue! I have searched and have seen that there might be some issues with the PortBinding method, but I was under the impression, that those issues only regard running tests in parallel, but for me it seems this issue also happens when test are run sequentially..

Any ideas / thoughts / comments? Thank you!

Cheers!

[HofmeisterAn]

I'm not sure if this is a bug or if I am doing something wrong?

This is not a bug. You cannot bind a host port more than once. If you create multiple containers and bind them to the same host port, the containers will fail to start. That's why we recommend using random assigned host ports: https://dotnet.testcontainers.org/api/create_docker_network/#exposing-container-ports-to-the-host. OneTimeSetUp will run once per fixture class, which in your example means it will run twice, once for TestClassOne and once for TestClassTwo. I hope that helps.

Edit: Simply do not assign fixed host ports. BTW, that's how modules are pre-configured. Your example simply needs:

_ = new PostgreSqlBuilder().Build();

[stefanteodoropol]

Hi, thanks for your fast reply!

I would really like to fix the port, so to allow for easier/faster debugging with external tools (i.E. DBeaver is configured to connect to the PostgresSql DB on a predefined port. It'a bit cumbersome to find the random port in Docker Desktop, then change the DBeaver config to connect to the DB on that new random port every time I want to debug a test.)

I was hoping I could "correctly" dispose of the container after a test fixture and let the port be available again for the next test class, is this not the expected result of the DisposeAsync() call?

[OneTimeTearDownAttribute]
public virtual async Task OneTimeTearDownAttribute()
{
    await _postgres.DisposeAsync();
}

[HofmeisterAn]

I was hoping I could "correctly" dispose of the container after a test fixture and let the port be available again for the next test class, is this not the expected result of the DisposeAsync() call?

Your test fixtures run in parallel. What you need is a singleton that sets up and tears down everything once for all test fixtures. However, this comes with the caveat that the test fixtures won't run in parallel anymore, and they could interfere with each other (the order of the tests could matter).

For local development, you can enable reuse, TC will use an existing container if it finds one. But, we still recommend using random assigned host ports for the final tests or using a proper singleton implementation like I mentioned before (still with a random assinged host port, since you can't be sure if the port is already taken on the host running the test).

[stefanteodoropol]

When viewing the Test Fixtures in the Visual Studio Test Explorer, they appear to run sequentially, and not in parallel.

According to the NUnit documentation also:

Parallelizable

Added in NUnit 3.7

The ParallelizableAttribute is used to indicate that a test and/or its descendants may be run in parallel with other tests. By default, no parallel execution takes place.

Additionally I tried adding the [NonParallelizable] attribute and also tried the [Parallelizable(ParallelScope.None)] attribute to the BaseTest class as well as to TestClassOne and TestClassTwo in case this wasn't the default, however the behavior was the same - the TestClassOne works, TestClassTwo fails with the exception.

I also tried adding .WithReuse(true), but still encountered the exception for TestClassTwo.

I also tried a Singleton variant of the BaseTest class and still encountered the same exception..

using Testcontainers.PostgreSql;

namespace TestContainer_Port_Bug_Demo
{
    [TestFixture]
    [Parallelizable(ParallelScope.None)] // Disable parallel execution for tests that extend this class
    public class BaseTest
    {
        protected static PostgreSqlContainer _postgres;
        protected static string _connectionString;

        private static readonly object _lock = new();
        private static bool _initialized = false;

        [OneTimeSetUp]
        public async Task OneTimeSetUp()
        {
            if (_initialized) return;

            lock (_lock)
            {
                if (_initialized) return;

                _postgres = new PostgreSqlBuilder()
                    .WithImage("postgres:alpine")
                    .WithPortBinding("3033", "5432") // Commenting this line out will let both tests pass.
                    .Build();
                _initialized = true;
            }

            await _postgres.StartAsync();
            _connectionString = _postgres.GetConnectionString();
        }

        [OneTimeTearDown]
        public async Task OneTimeTearDown()
        {
            lock (_lock)
            {
                if (!_initialized) return;

                _initialized = false;
            }

            if (_postgres != null)
            {
                await _postgres.DisposeAsync();
                _postgres = null;
            }
        }
    }
}

For my debugging environment, it is fine that the tests don't run in parallel, however I cannot get them to "let go" of the port after the tests have run. I would really prefer to bind the Test Containers to the same port, spinning up a fresh container per Test Fixture, and then properly disposing it after all tests in a Test Fixture have run. - Freeing up the port for the next Test Fixture..

Is there anything else I can try?

[HofmeisterAn]

According to the NUnit documentation also:

I see, then test fixtures behave differently than the collections in xUnit.net. I haven't used NUnit in a long time (and just assumed that).

I checked out your reproducer and took a look at it. It seems that the PostgreSQL client library caches the connection (using a connection pool). I'm not sure if there are other ways to disable pooling, but you can clear the pool using:

NpgsqlConnection.ClearAllPools();

I noticed another issue in your reproducer. Remove the [TestFixture] attribute from the base class. Here's my updated diff that works:

diff --git a/TestContainer_Port_Bug_Demo/BaseTest.cs b/TestContainer_Port_Bug_Demo/BaseTest.cs
index bbbb6c6..89a2709 100644
--- a/TestContainer_Port_Bug_Demo/BaseTest.cs
+++ b/TestContainer_Port_Bug_Demo/BaseTest.cs
@@ -1,14 +1,18 @@
+using Npgsql;
 using Testcontainers.PostgreSql;
 
 namespace TestContainer_Port_Bug_Demo
 {
-    [TestFixture]
     public class BaseTest
     {
         protected PostgreSqlContainer _postgres;
 
         protected string _connectionString;
-
+        
+        public BaseTest()
+        {
+            NpgsqlConnection.ClearAllPools();
+        }
 
         [OneTimeSetUpAttribute]
         public virtual async Task OneTimeSetUpAttribute()