Using Windows Firewall and IPSEC to secure Docker Connection

[ntrappe-msft]

Important

Migrating Discussions to Issues. All customer inquiries should be in Issues.

Discussed in https://github.com/microsoft/Windows-Containers/discussions/453

^{Originally posted by brwilkinson January 3, 2024}
I have been able to create a IPSEC Rule with Windows Firewall.

Get-NetIPsecRule

IPsecRuleName         : IPSEC-TCP-2375
DisplayName           : IPSEC-TCP-2375
Description           :
DisplayGroup          :
Group                 :
Enabled               : True
Profile               : Any
Platform              : {}
Mode                  : Transport
InboundSecurity       : Request
OutboundSecurity      : Request
QuickModeCryptoSet    : Default
Phase1AuthSet         : ComputerKerberos
Phase2AuthSet         : UserKerberos
KeyModule             : Default
AllowWatchKey         : False
AllowSetKey           : False
LocalTunnelEndpoint   :
RemoteTunnelEndpoint  :
RemoteTunnelHostname  :
ForwardPathLifetime   : 0
EncryptedTunnelBypass : False
RequireAuthorization  : False
User                  : Any
Machine               : Any
PrimaryStatus         : OK
Status                : The rule was parsed successfully from the store. (65536)
EnforcementStatus     : NotApplicable
PolicyStoreSource     : PersistentStore
PolicyStoreSourceType : Local

This used Computer kerberos for phase 1 (main mode) and User kerberos for Phase 1 auth.

Which allows for a Secure association between 2 host computers via IPSec using NOEncap.

Get-NetIPsecMainModeSA

Name                                : 4777
LocalEndpoint                       : 10.4.169.112
RemoteEndpoint                      : 10.4.170.183
LocalFirstId.Identity               : host/server001.contoso.com
LocalFirstId.ImpersonationType      : None
LocalFirstId.AuthenticationMethod   : Kerberos
LocalFirstId.Flags                  : None
RemoteFirstId.Identity              : contoso\server002$
RemoteFirstId.ImpersonationType     : None
RemoteFirstId.AuthenticationMethod  : Kerberos
RemoteFirstId.Flags                 : None
LocalSecondId.Identity              : host/server001.contoso.com
LocalSecondId.ImpersonationType     : None
LocalSecondId.AuthenticationMethod  : Kerberos
LocalSecondId.Flags                 : None
RemoteSecondId.Identity             : contoso\svcmygMSA$
RemoteSecondId.ImpersonationType    : Impersonated
RemoteSecondId.AuthenticationMethod : Kerberos
RemoteSecondId.Flags                : None
CipherAlgorithm                     : AES128
HashAlgorithm                       : SHA1
GroupId                             : None
KeyModule                           : AuthIP
MaxQMSAs                            : 0
LifetimeSeconds                     : 28800
LocalUdpEncapsulationPort           :
RemoteUdpEncapsulationPort          :
ExtendedFilterId                    : 0

I have configured Docker with... (since it will be protected by IPSEC and the Windows Firewall)

{
    "hosts": ["npipe://", "tcp://0.0.0.0:2375"]
}

I have created a Firewall rule that supports authenticated connections with noEncap.

Get-NetFirewallRule | where DisplayName -match docker

Name                          : DSC-Allow-Docker-2375-Authenticated
DisplayName                   : DSC-Allow-Docker-2375-Authenticated
Description                   : DSC-Allow-Docker-2375-Authenticated
DisplayGroup                  :
Group                         :
Enabled                       : True
Profile                       : Any
Platform                      : {}
Direction                     : Inbound
Action                        : Allow
EdgeTraversalPolicy           : Block
LooseSourceMapping            : False
LocalOnlyMapping              : False
Owner                         :
PrimaryStatus                 : OK
Status                        : The rule was parsed successfully from the store. (65536)
EnforcementStatus             : NotApplicable
PolicyStoreSource             : PersistentStore
PolicyStoreSourceType         : Local
RemoteDynamicKeywordAddresses : {}
PolicyAppId

Which is mapped to the Security filter on the firewall rule...

Get-NetFirewallSecurityFilter -Authentication NoEncap

Authentication     : NoEncap
Encryption         : NotRequired
OverrideBlockRules : False
LocalUser          : Any
RemoteUser         : O:LSD:(A;;CC;;;S-1-5-X-MYSID)
RemoteMachine      : O:LSD:(A;;CC;;;S-1-5-X-MYSID)
Authentication     : NoEncap
Encryption         : NotRequired
OverrideBlockRules : False
LocalUser          : Any
RemoteUser         : Any
RemoteMachine      : Any

# this work great for Server to Docker communication 

docker info
> return as expected via the tunnel and the firewall rule allows the traffic

invoke-restmethod -uri http://server001:2375/v1.24/info
> returns the expected results via the tunnel.

This process is documented in many places such as below.

https://learn.microsoft.com/en-us/windows/security/operating-system-security/network-security/windows-firewall/configure-with-command-line?tabs=powershell#create-ipsec-rules

https://learn.microsoft.com/en-us/training/modules/explore-connection-security-rules/

So now my question or discussion point..

I would like to initiate the docker connection from inside a container running on a remote windows container host.

• Currently I am not sure how to initiate the "Secure Association"

1. if I open port with a standard firewall rule, then the remote container/docker command works fine.
2. however I need to filter it to the remote computer with the ipsec authenticated rule.

The process tree is as follows:

• wininit.exe
  • services.exe (Below here is running inside the container)
    • CExecSVC.exe
      • pwsh.exe
        • java.exe --> this web app is what initiates the docker commands to the remove containers protected by IPSEC

If I run the same java.exe process outside of a container it works great. Both that service and the container are both configured to run under a group managed service account, which is allowed through the remote firewall rule as part of the security filter. I would like to run both ends inside of a windows container with process isolation (not hyer-v isolation, at least at this time).

I have just left the default NAT networking for the containers, since that has been working fine un until now.

I was wondering if I could switch to L2Bridge as an alternative in an attempt to get the IPSEC secure association being made? I haven't tested this as yet.

• https://learn.microsoft.com/en-us/virtualization/windowscontainers/container-networking/network-drivers-topologies#networking-modesdocker-drivers

It is called out here that Host mode networking and Encrypted container communication is not supported.

• However I would like for the IPSEC to occur at the host level not the container level. I just need to initiate the secure association, then I guess have the traffic route via the ipsec tunnel, which in this case is just signing the traffic.

• https://learn.microsoft.com/en-us/virtualization/windowscontainers/container-networking/architecture#unsupported-features-and-network-options

Unsupported features and network options

• The following networking options are currently NOT supported on Windows:

  • From Windows Server 2022 onwards, Windows containers attached to l2bridge networks support the IPv6 stack. However, Windows containers attached to NAT and overlay networks do not support communicating over the IPv6 stack.
  • Encrypted container communication via IPsec.
  • Host mode networking.
  • Networking on virtualized Azure infrastructure via the transparent network driver.

The main driver behind this is because I don't want to manage the TLS certs to secure the Docker communication.

e.g. The following alternate configuration.

{
    "hosts": ["npipe://", "tcp://0.0.0.0:2376"],
    "tlsverify": true,
    "tlscacert": "C:/ProgramData/docker/config/ca.pem",
    "tlscert": "C:/ProgramData/docker/config/{0}-serverAuth-cert.pem",
    "tlskey": "C:/ProgramData/docker/config/{0}-serverAuth-key.pem"
}

[github-actions]

Thank you for creating an Issue. Please note that GitHub is not an official channel for Microsoft support requests. To create an official support request, please open a ticket here. Microsoft and the GitHub Community strive to provide a best effort in answering questions and supporting Issues on GitHub.