[SC-7] Restrict egress traffic from EKS workloads by default, allow by configuration

[mogul]

User Story

In order to meet the intent of SC-7 (and sub-controls), EKS clusters provisioned by the SSB should restrict outbound traffic by default, and allow by exception/explicit configuration.

Acceptance Criteria

[ACs should be clearly demoable/verifiable whenever possible. Try specifying them using BDD.]

Outbound:

• GIVEN I have provisioned an EKS cluster through the SSB
  AND I did not configure allowed outbound CIDRs during provision
  WHEN I kubectl ssh into a pod
  AND try to ping well-known public IPs
  THEN the traffic is blocked.
• GIVEN I have provisioned an EKS cluster through the SSB
  AND I configured allowed outbound CIDRs during provision
  WHEN I kubectl ssh into a pod
  AND try to ping an IP in those allowed CIDR ranges
  THEN the traffic is permitted.

Background

[Any helpful contextual notes or links to artifacts/evidence, if needed]
NIST SC-7 says outbound traffic should be permitted by exception, not by default. We can restrict outbound IP ranges to just those configured at provisioning (or none, if none were provided) to meet the intent of this control for the SSB's managed boundary.

Security Considerations (required)

[Any security concerns that might be implicated in the change. "None" is OK, just be explicit here!]

Sketch

Most flexible option for users of the broker (though more complex and not a core k8s feature):

• Implement hostname-based egress proxying by deploying the Monzo egress-operator.

The simplest-to-implement near-term alternative is to just implementing CIDR-based controls:

• Deploy a default-deny k8s NetworkPolicy in each requested namespace.

[mogul]

Sketch

[...]
Two alternative options for implementing CIDR-based controls:

[...]

• Deploy a default-deny k8s NetworkPolicy in each requested namespace. (customizable per-namespace)

I said in our meeting today that I thought the VPC-CNI add-on for EKS would be sufficient for us to implement a default-deny NetworkPolicy, but I was wrong... We still need Calico to do that. Luckily installing and validating Calico is pretty easy these days.

[mogul]

Note that the monzo-operator hasn't been touched in 11 months, so it may no longer be a first-choice solution.

[nickumia-reisys]

Resources Used:

• AWS Install Calico
• Calico Helm Instructions
• Terraform kubernetes_namespace
• Terraform kubernetes_network_policy

[nickumia-reisys]

There's no way to implement a network policy that applies to all namespaces, so we must do them individually 😢

https://kubernetes.io/docs/concepts/services-networking/network-policies/#what-you-can-t-do-with-network-policies-at-least-not-yet

[mogul]

That's fine, we can do that with a for_each resource. Besides, right now we're brokering only one namespace, default.

[mogul]

For future reference: Replacing Calico with Cilium would give us DNS-based egress control (like the Monzo operator) and mTLS between pods as well as many other service-mesh features.
https://cilium.io/