handling selectors with matchexpressions (fixed) (#377)

* support match expression operators for generating and selecting representative peers + first examples * more tests * more tests * updating code with label selectors * merge fixes * duplicated tests with matching pods * fixing code + tests with multiple policies * update comments in exposure.go * renaming function and updating comments and doc of representative_selectors.go * move `RepresentativeNsLabelSelector` field from namespace.go to pod.go * 1. reverting changes to AddPodByNameAndNamespace and resolveSingleMissingNamespace (to original version from main branch) 2. creating a new func for adding representative pods to the policy-engine, without representative namespaces. a representative pod which should not be in a real namespace, will have no namespace * avoid duplicating code of generating the default namespace name map; and some updates to netpol.go * eliminate representativePeer.PotentialNamespaceLabelSelector as it duplicates Pod.RepresentativeNsLabelSelector * renaming the func in representative_selectors.go again * a new test with handling a special case of equiv rules written in a different way * unit test for representative_selectors.go * removing redundant code * updating documentation of new fields in pod.go * fixes in resources.go * fix in check.go * update few comments Signed-off-by: adisos <[email protected]> * renaming AddObjects + updating its documentation * renaming netpol funcs * renaming connPeers * fixing representative pods naming and updating relevant funcs * renaming "GeneralConns" to "ExposedGeneralConns" * removing PolicyNsFlag * no need to split namespaces with policies at first * Revert "no need to split namespaces with policies at first" This reverts commit 03e384e. * rename extractLabelsAndRefineRepresentativePeers and refineRepresentativePeersMatchingLabels * renaming checkIfP2PConnOrExposureConn * lint fix * func allAllowedConnectionsBetweenPeers: remove ingressSet, egressSet * using new terms for general conns : ClusterWideExposure and ExternalExposure * an example why should split namespaces at the beginning with the policies * eliminate RepresentativePeer struct * fixing some typos and adding some very used words to a cspell file * more typos fixes * updating some comments * updating readme (all formats supported) * getting netpols before pods for live cluster - so it works well for both exposure-analysis on/off * Update pkg/netpol/eval/check.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/check.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/exposure.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/exposure.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/internal/k8s/netpol.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/internal/k8s/netpol.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/internal/k8s/netpol.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/internal/k8s/netpol.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/internal/k8s/netpol.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/exposure.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/exposure.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/exposure.go Co-authored-by: Adi Sosnovich <[email protected]> * rename getSelectorsAndUpdateExposedGeneralConns * rename ScanPolicyRulesAndUpdateExposedWideConns * rename updateNetworkPolicyWideExposureConns * Update pkg/netpol/eval/resources.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/internal/k8s/peer.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/internal/k8s/peer.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/internal/k8s/pod.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/internal/k8s/pod.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/resources.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/resources.go Co-authored-by: Adi Sosnovich <[email protected]> * fixing lint * Update pkg/netpol/eval/internal/k8s/representative_selectors.go Co-authored-by: Adi Sosnovich <[email protected]> * lint fix * Update pkg/netpol/eval/internal/k8s/representative_selectors.go Co-authored-by: Adi Sosnovich <[email protected]> * fixing the last commit * fixing the SelectorsFullMatch doc * removing unnecessaryDeepCopy calls * Update pkg/netpol/eval/internal/k8s/netpol.go Co-authored-by: Adi Sosnovich <[email protected]> * Update pkg/netpol/eval/internal/k8s/netpol.go Co-authored-by: Adi Sosnovich <[email protected]> * lint fix * some renamings in representative_selectors + document why returning full match for rep selector in case of empty rule * adding line to comment * split funcs in check.go for readability * rename hasRepresentativePod * updating comment * updating comment of storing the named port * updating String() func of workloadpeer * comment update * updating comment * new func of selectors match in `netpol.go` to avoid duplicates * updating comment in pod.go (what do the combinations of rep selectors imply for) * renaming str vars * eliminating addIfMissingNamespace func * new tests - rep peers when there is real ns but no real pods matching * add comment on String() func * rename handleRequirementWithInOpAndSingleValue * renaming test dirs and expected output of exposure-analysis tests * new fixes --------- Signed-off-by: adisos <[email protected]> Co-authored-by: adisos <[email protected]> Co-authored-by: Adi Sosnovich <[email protected]>
np-guard · Aug 8, 2024 · 796ae5f · 796ae5f
1 parent 5c6b797
commit 796ae5f
Show file tree

Hide file tree

Showing 606 changed files with 13,733 additions and 960 deletions.
diff --git a/README.md b/README.md
@@ -64,7 +64,7 @@ Flags:
   -f, --file string            Write output to specified file
       --focusworkload       Focus connections of specified workload in the output (supported formats: <workload-name>, <workload-namespace>/<workload-name>)
                             (to focus connections from Ingress/Route only, use `ingress-controller` as <workload-name>)
-  -o, --output string       Required output format (txt, json, dot, csv, md) or (txt, dot) with exposure analysis (default "txt")
+  -o, --output string       Required output format (txt, json, dot, csv, md) (default "txt")
   -h, --help   help for list
 
 Global Flags:

diff --git a/cspell.json b/cspell.json
@@ -0,0 +1,15 @@
+{
+    "version": "0.2",
+    "ignorePaths": [],
+    "dictionaryDefinitions": [],
+    "dictionaries": [],
+    "words": [
+        "connlist",
+        "netpol",
+        "netpols",
+        "SCTP",
+        "xgress"
+    ],
+    "ignoreWords": [],
+    "import": []
+}
diff --git a/pkg/cli/diff.go b/pkg/cli/diff.go
@@ -34,8 +34,8 @@ func runDiffCommand() error {
 	var connsDiff diff.ConnectivityDiff
 	var err error
 
-	clogger := logger.NewDefaultLoggerWithVerbosity(detrmineLogVerbosity())
-	diffAnalyzer := diff.NewDiffAnalyzer(getDiffOptions(clogger)...)
+	cLogger := logger.NewDefaultLoggerWithVerbosity(determineLogVerbosity())
+	diffAnalyzer := diff.NewDiffAnalyzer(getDiffOptions(cLogger)...)
 
 	connsDiff, err = diffAnalyzer.ConnDiffFromDirPaths(dir1, dir2)
 	if err != nil {

diff --git a/pkg/cli/evaluate.go b/pkg/cli/evaluate.go
@@ -50,7 +50,7 @@ func validateEvalFlags() error {
 	if destinationPod.Name == "" && dstExternalIP == "" {
 		return errors.New(netpolerrors.NoDestDefinedErr)
 	} else if destinationPod.Name != "" && dstExternalIP != "" {
-		return errors.New(netpolerrors.OnlyOneDstFalgErrStr)
+		return errors.New(netpolerrors.OnlyOneDstFlagErrStr)
 	}
 
 	if srcExternalIP != "" && dstExternalIP == "" {
@@ -65,22 +65,22 @@ func validateEvalFlags() error {
 
 func updatePolicyEngineObjectsFromDirPath(pe *eval.PolicyEngine, podNames []types.NamespacedName) error {
 	// get relevant resources from dir path
-	elogger := logger.NewDefaultLoggerWithVerbosity(detrmineLogVerbosity())
+	eLogger := logger.NewDefaultLoggerWithVerbosity(determineLogVerbosity())
 
 	rList, errs := fsscanner.GetResourceInfosFromDirPath([]string{dirPath}, true, false)
 	if errs != nil {
 		// TODO: consider avoid logging this error because it is already printed to log by the builder
 		if len(rList) == 0 || stopOnFirstError {
 			err := utilerrors.NewAggregate(errs)
-			elogger.Errorf(err, netpolerrors.ErrGettingResInfoFromDir)
+			eLogger.Errorf(err, netpolerrors.ErrGettingResInfoFromDir)
 			return err // return as fatal error if rList is empty or if stopOnError is on
 		}
 		// split err if it's an aggregated error to a list of separate errors
 		for _, err := range errs {
-			elogger.Errorf(err, netpolerrors.FailedReadingFileErrorStr) // print to log the error from builder
+			eLogger.Errorf(err, netpolerrors.FailedReadingFileErrorStr) // print to log the error from builder
 		}
 	}
-	objectsList, processingErrs := parser.ResourceInfoListToK8sObjectsList(rList, elogger, false)
+	objectsList, processingErrs := parser.ResourceInfoListToK8sObjectsList(rList, eLogger, false)
 	for _, err := range processingErrs {
 		if err.IsFatal() || (stopOnFirstError && err.IsSevere()) {
 			return fmt.Errorf("scan dir path %s had processing errors: %w", dirPath, err.Error())
@@ -92,11 +92,11 @@ func updatePolicyEngineObjectsFromDirPath(pe *eval.PolicyEngine, podNames []type
 	for _, obj := range objectsList {
 		switch obj.Kind {
 		case parser.Pod:
-			err = pe.UpsertObject(obj.Pod)
+			err = pe.InsertObject(obj.Pod)
 		case parser.Namespace:
-			err = pe.UpsertObject(obj.Namespace)
-		case parser.Networkpolicy:
-			err = pe.UpsertObject(obj.Networkpolicy)
+			err = pe.InsertObject(obj.Namespace)
+		case parser.NetworkPolicy:
+			err = pe.InsertObject(obj.NetworkPolicy)
 		default:
 			continue
 		}
@@ -114,32 +114,32 @@ func updatePolicyEngineObjectsFromLiveCluster(pe *eval.PolicyEngine, podNames []
 	defer cancel()
 
 	for _, name := range nsNames {
-		ns, apierr := clientset.CoreV1().Namespaces().Get(ctx, name, metav1.GetOptions{})
-		if apierr != nil {
-			return apierr
+		ns, apiErr := clientset.CoreV1().Namespaces().Get(ctx, name, metav1.GetOptions{})
+		if apiErr != nil {
+			return apiErr
 		}
-		if err := pe.UpsertObject(ns); err != nil {
+		if err := pe.InsertObject(ns); err != nil {
 			return err
 		}
 	}
 
 	for _, name := range podNames {
-		pod, apierr := clientset.CoreV1().Pods(name.Namespace).Get(ctx, name.Name, metav1.GetOptions{})
-		if apierr != nil {
-			return apierr
+		pod, apiErr := clientset.CoreV1().Pods(name.Namespace).Get(ctx, name.Name, metav1.GetOptions{})
+		if apiErr != nil {
+			return apiErr
 		}
-		if err := pe.UpsertObject(pod); err != nil {
+		if err := pe.InsertObject(pod); err != nil {
 			return err
 		}
 	}
 
 	for _, ns := range nsNames {
-		npList, apierr := clientset.NetworkingV1().NetworkPolicies(ns).List(ctx, metav1.ListOptions{})
-		if apierr != nil {
-			return apierr
+		npList, apiErr := clientset.NetworkingV1().NetworkPolicies(ns).List(ctx, metav1.ListOptions{})
+		if apiErr != nil {
+			return apiErr
 		}
 		for i := range npList.Items {
-			if err := pe.UpsertObject(&npList.Items[i]); err != nil {
+			if err := pe.InsertObject(&npList.Items[i]); err != nil {
 				return err
 			}
 		}

diff --git a/pkg/cli/list.go b/pkg/cli/list.go
@@ -35,8 +35,8 @@ func runListCommand() error {
 	var conns []connlist.Peer2PeerConnection
 	var err error
 
-	clogger := logger.NewDefaultLoggerWithVerbosity(detrmineLogVerbosity())
-	analyzer := connlist.NewConnlistAnalyzer(getConnlistOptions(clogger)...)
+	cLogger := logger.NewDefaultLoggerWithVerbosity(determineLogVerbosity())
+	analyzer := connlist.NewConnlistAnalyzer(getConnlistOptions(cLogger)...)
 
 	if dirPath != "" {
 		conns, _, err = analyzer.ConnlistFromDirPath(dirPath)

diff --git a/pkg/cli/root.go b/pkg/cli/root.go
@@ -32,7 +32,7 @@ var (
 )
 
 // returns verbosity level based on the -q and -v switches
-func detrmineLogVerbosity() logger.Verbosity {
+func determineLogVerbosity() logger.Verbosity {
 	verbosity := logger.DefaultVerbosity
 	if quiet {
 		verbosity = logger.LowVerbosity

diff --git a/pkg/internal/netpolerrors/netpol_errors.go b/pkg/internal/netpolerrors/netpol_errors.go
@@ -41,15 +41,15 @@ const (
 	NotFoundNamespace      = "could not find peer namespace"
 	OnlyOneSrcFlagErrStr   = "only one of source pod and namespace or external IP can be defined, not both"
 	NoDestDefinedErr       = "no destination defined, destination pod and namespace or external IP required"
-	OnlyOneDstFalgErrStr   = "only one of destination pod and namespace or external IP can be defined, not both"
+	OnlyOneDstFlagErrStr   = "only one of destination pod and namespace or external IP can be defined, not both"
 	OnlyOneIPPeerErrStr    = "only one of source or destination can be defined as external IP, not both"
 	RequiredDstPortFlagErr = "destination port name or value is required"
 
 	// diff command errors
 	RequiredFlagsErr = "both directory paths dir1 and dir2 are required"
 	FlagMisUseErr    = "dirpath flag is not used with diff command"
 
-	// errors consts from `orig errors` that are raised by external libraries
+	// errors constants from `orig errors` that are raised by external libraries
 	InvalidCIDRAddr         = "invalid CIDR address"
 	InvalidKeyVal           = "key: Invalid value"
 	UnrecognizedValType     = "unrecognized type"
@@ -62,6 +62,10 @@ const (
 	UnableToDecodeErr       = "unable to decode"
 
 	UnknownCommandErr = "unknown command"
+
+	NilRepresentativePodSelectorsErr = "representative pod might not be generated if it does not have any representative selector"
+	NilNamespaceAndNilNsSelectorErr  = "representative pod might not be generated from nil namespace-selector and nil namespace;" +
+		"at least one should not be nil"
 )
 
 // NotSupportedPodResourcesErrorStr returns error string of not supported pods with same ownerRef but different labels
@@ -70,7 +74,7 @@ func NotSupportedPodResourcesErrorStr(ownerRefName string) string {
 		ownerRefName + " but with different set of labels."
 }
 
-// WorkloadDoesNotExistErrStr returns error string of missing workload for connlist with focusworkload
+// WorkloadDoesNotExistErrStr returns error string of missing workload for connlist with focus-workload
 func WorkloadDoesNotExistErrStr(workload string) string {
 	return "Workload " + workload + " does not exist in the input resources." + EmptyConnListErrStr
 }
@@ -98,8 +102,8 @@ func BlockedIngressWarning(objKind, objName, peerStr string) string {
 }
 
 // MissingNamespaceErrStr returns error string of a missing namespace of a peer
-func MissingNamespaceErrStr(peerStr string) string {
-	return "error: namespace of pod " + peerStr + " is missing"
+func MissingNamespaceErrStr(nsName, peerName string) string {
+	return "error: namespace " + nsName + " of pod " + peerName + " is missing"
 }
 
 // NotPeerErrStr returns error string of a peer that is not workload peer

diff --git a/pkg/internal/testutils/testutils.go b/pkg/internal/testutils/testutils.go
@@ -26,6 +26,7 @@ var update = flag.Bool("update", false, "write or override golden files")
 
 const (
 	connlistExpectedOutputFilePartialName = "connlist_output."
+	exposureExpectedOutputFilePartialName = "exposure_output."
 	underscore                            = "_"
 	dotSign                               = "."
 	formatStr                             = "_format_"
@@ -43,15 +44,15 @@ func GetTestDirPath(dirName string) string {
 // ConnlistTestNameByTestArgs returns connlist test name and test's expected output file from some tests args
 func ConnlistTestNameByTestArgs(dirName, focusWorkload, format string, exposureFlag bool) (testName, expectedOutputFileName string) {
 	namePrefix := dirName
-	if exposureFlag {
-		// if dir name contains a separator; last element is enough for the test and file names
-		namePrefix = "exposure_" + filepath.Base(dirName)
-	}
 	if focusWorkload != "" {
 		namePrefix += focusWlAnnotation + strings.Replace(focusWorkload, "/", underscore, 1)
 	}
 	testName = namePrefix + formatStr + format
-	expectedOutputFileName = namePrefix + underscore + connlistExpectedOutputFilePartialName + format
+	outputPartialName := connlistExpectedOutputFilePartialName
+	if exposureFlag {
+		outputPartialName = exposureExpectedOutputFilePartialName
+	}
+	expectedOutputFileName = namePrefix + underscore + outputPartialName + format
 	return testName, expectedOutputFileName
 }
 

diff --git a/pkg/manifests/parser/k8sobj.go b/pkg/manifests/parser/k8sobj.go
@@ -19,19 +19,19 @@ import (
 
 // relevant K8s resource kinds as string values
 const (
-	Networkpolicy         string = "NetworkPolicy"
+	NetworkPolicy         string = "NetworkPolicy"
 	Namespace             string = "Namespace"
 	Pod                   string = "Pod"
 	ReplicaSet            string = "ReplicaSet"
 	ReplicationController string = "ReplicationController"
 	Deployment            string = "Deployment"
-	Statefulset           string = "StatefulSet"
-	Daemonset             string = "DaemonSet"
+	StatefulSet           string = "StatefulSet"
+	DaemonSet             string = "DaemonSet"
 	Job                   string = "Job"
 	CronJob               string = "CronJob"
 	List                  string = "List"
 	NamespaceList         string = "NamespaceList"
-	NetworkpolicyList     string = "NetworkPolicyList"
+	NetworkPolicyList     string = "NetworkPolicyList"
 	PodList               string = "PodList"
 	Service               string = "Service"
 	Route                 string = "Route"
@@ -45,7 +45,7 @@ type K8sObject struct {
 	Namespace *v1.Namespace
 
 	// netpol object
-	Networkpolicy *netv1.NetworkPolicy
+	NetworkPolicy *netv1.NetworkPolicy
 
 	// pod object
 	Pod *v1.Pod
@@ -58,29 +58,29 @@ type K8sObject struct {
 	Ingress *netv1.Ingress
 
 	// workload object
-	Replicaset            *appsv1.ReplicaSet
+	ReplicaSet            *appsv1.ReplicaSet
 	Deployment            *appsv1.Deployment
-	Statefulset           *appsv1.StatefulSet
+	StatefulSet           *appsv1.StatefulSet
 	ReplicationController *v1.ReplicationController
 	Job                   *batchv1.Job
 	CronJob               *batchv1.CronJob
-	Daemonset             *appsv1.DaemonSet
+	DaemonSet             *appsv1.DaemonSet
 }
 
 func (k *K8sObject) getEmptyInitializedFieldObjByKind(kind string) interface{} {
 	switch kind {
 	case Deployment:
 		k.Deployment = &appsv1.Deployment{}
 		return k.Deployment
-	case Daemonset:
-		k.Daemonset = &appsv1.DaemonSet{}
-		return k.Daemonset
+	case DaemonSet:
+		k.DaemonSet = &appsv1.DaemonSet{}
+		return k.DaemonSet
 	case ReplicaSet:
-		k.Replicaset = &appsv1.ReplicaSet{}
-		return k.Replicaset
-	case Statefulset:
-		k.Statefulset = &appsv1.StatefulSet{}
-		return k.Statefulset
+		k.ReplicaSet = &appsv1.ReplicaSet{}
+		return k.ReplicaSet
+	case StatefulSet:
+		k.StatefulSet = &appsv1.StatefulSet{}
+		return k.StatefulSet
 	case ReplicationController:
 		k.ReplicationController = &v1.ReplicationController{}
 		return k.ReplicationController
@@ -102,9 +102,9 @@ func (k *K8sObject) getEmptyInitializedFieldObjByKind(kind string) interface{} {
 	case Pod:
 		k.Pod = &v1.Pod{}
 		return k.Pod
-	case Networkpolicy:
-		k.Networkpolicy = &netv1.NetworkPolicy{}
-		return k.Networkpolicy
+	case NetworkPolicy:
+		k.NetworkPolicy = &netv1.NetworkPolicy{}
+		return k.NetworkPolicy
 	case Namespace:
 		k.Namespace = &v1.Namespace{}
 		return k.Namespace
@@ -119,17 +119,17 @@ func (k *K8sObject) initDefaultNamespace() {
 		if k.Deployment.Namespace == "" {
 			k.Deployment.Namespace = metav1.NamespaceDefault
 		}
-	case Daemonset:
-		if k.Daemonset.Namespace == "" {
-			k.Daemonset.Namespace = metav1.NamespaceDefault
+	case DaemonSet:
+		if k.DaemonSet.Namespace == "" {
+			k.DaemonSet.Namespace = metav1.NamespaceDefault
 		}
 	case ReplicaSet:
-		if k.Replicaset.Namespace == "" {
-			k.Replicaset.Namespace = metav1.NamespaceDefault
+		if k.ReplicaSet.Namespace == "" {
+			k.ReplicaSet.Namespace = metav1.NamespaceDefault
 		}
-	case Statefulset:
-		if k.Statefulset.Namespace == "" {
-			k.Statefulset.Namespace = metav1.NamespaceDefault
+	case StatefulSet:
+		if k.StatefulSet.Namespace == "" {
+			k.StatefulSet.Namespace = metav1.NamespaceDefault
 		}
 	case ReplicationController:
 		if k.ReplicationController.Namespace == "" {
@@ -160,9 +160,9 @@ func (k *K8sObject) initDefaultNamespace() {
 			k.Pod.Namespace = metav1.NamespaceDefault
 		}
 		checkAndUpdatePodStatusIPsFields(k.Pod)
-	case Networkpolicy:
-		if k.Networkpolicy.Namespace == "" {
-			k.Networkpolicy.Namespace = metav1.NamespaceDefault
+	case NetworkPolicy:
+		if k.NetworkPolicy.Namespace == "" {
+			k.NetworkPolicy.Namespace = metav1.NamespaceDefault
 		}
 	}
 }
@@ -184,8 +184,8 @@ var workloadKinds = map[string]bool{
 	Pod:                   true,
 	ReplicaSet:            true,
 	Deployment:            true,
-	Statefulset:           true,
-	Daemonset:             true,
+	StatefulSet:           true,
+	DaemonSet:             true,
 	Job:                   true,
 	CronJob:               true,
 	ReplicationController: true,
@@ -205,8 +205,8 @@ func FilterObjectsList(allObjects []K8sObject, podNames []types.NamespacedName)
 			if _, ok := nsMap[obj.Namespace.Name]; ok {
 				res = append(res, obj)
 			}
-		case Networkpolicy:
-			if _, ok := nsMap[obj.Networkpolicy.Namespace]; ok {
+		case NetworkPolicy:
+			if _, ok := nsMap[obj.NetworkPolicy.Namespace]; ok {
 				res = append(res, obj)
 			}
 		case Pod:

diff --git a/pkg/manifests/parser/parser.go b/pkg/manifests/parser/parser.go
@@ -36,7 +36,7 @@ func ResourceInfoListToK8sObjectsList(infosList []*resource.Info, l logger.Logge
 		}
 		if k8sObj != nil && k8sObj.Kind != "" {
 			res = append(res, *k8sObj)
-			if k8sObj.Kind == Networkpolicy {
+			if k8sObj.Kind == NetworkPolicy {
 				hasNetpols = true
 			}
 			if workloadKinds[k8sObj.Kind] {