Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

98 grouping with self loops #152

Merged
merged 55 commits into from
Sep 28, 2023
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
ee600cf
So that grouping can be reused by subnetConnectivity
ShiriMoran Aug 14, 2023
43e8d41
VPCsubnetConnectivity computed and stored on EndpointElem instead of …
ShiriMoran Aug 14, 2023
c3f5519
Merge remote-tracking branch 'origin/main'
ShiriMoran Aug 17, 2023
2eb1ef1
Merge remote-tracking branch 'origin/main'
ShiriMoran Aug 22, 2023
df04274
fix comment (should have been part of a previous branch)
ShiriMoran Aug 22, 2023
e202533
extracting code to a function to be reused
ShiriMoran Aug 22, 2023
b9f3d3f
grouping with self loops - the first stages
ShiriMoran Aug 22, 2023
b97263d
commit before context switching to https://github.com/np-guard/vpc-ne…
ShiriMoran Aug 23, 2023
abb8c03
Merge remote-tracking branch 'origin/main' into 138_grouping_with_sel…
ShiriMoran Aug 24, 2023
fe16a2d
code for computing delta between GroupedConnLine
ShiriMoran Aug 24, 2023
7df7c1d
mergeGroups computed
ShiriMoran Aug 27, 2023
237f70b
merge groups to be merged
ShiriMoran Sep 5, 2023
82d7c34
vsis should be grouped only if from the same subnet
ShiriMoran Sep 5, 2023
af7e0ab
update testing
ShiriMoran Sep 5, 2023
6897003
fixed comment
ShiriMoran Sep 5, 2023
24e903e
lint comments #1
ShiriMoran Sep 5, 2023
2deebf5
lint comments #2
ShiriMoran Sep 5, 2023
be56d4d
lint comments #3
ShiriMoran Sep 5, 2023
d359811
lint comments #4
ShiriMoran Sep 5, 2023
a35a6fc
moved grouping self loops related code into a separate file
ShiriMoran Sep 12, 2023
0345421
optimization: consider only groups which are of internal source and d…
ShiriMoran Sep 12, 2023
1ebd67a
returning deleted by mistake
ShiriMoran Sep 13, 2023
afe2247
compute buckets
ShiriMoran Sep 13, 2023
e36112e
heuristic code written
ShiriMoran Sep 13, 2023
1212e6f
groupsToBeMerge uses heuristic
ShiriMoran Sep 13, 2023
5db16fd
renaming
ShiriMoran Sep 14, 2023
6cb4900
bug fix
ShiriMoran Sep 14, 2023
946ecc4
redundant function
ShiriMoran Sep 14, 2023
1276933
improved alg description
ShiriMoran Sep 14, 2023
878382f
unit tests for grouping; the last one fails. Big must be fixed
ShiriMoran Sep 14, 2023
a8a06b1
bug (typo) fix
ShiriMoran Sep 18, 2023
f65a63a
finalizing test5
ShiriMoran Sep 18, 2023
744ac97
Merge branch 'main' into 98_grouping_with_self_loops
ShiriMoran Sep 18, 2023
1388ee2
adding another unit test
ShiriMoran Sep 18, 2023
0deeb4e
Merge branch 'main' into 98_grouping_with_self_loops
ShiriMoran Sep 20, 2023
3ad19b4
initial review comments
ShiriMoran Sep 20, 2023
d9f7a0c
Merge remote-tracking branch 'origin/98_grouping_with_self_loops' int…
ShiriMoran Sep 20, 2023
ad43844
Merge remote-tracking branch 'origin/main' into 98_grouping_with_self…
ShiriMoran Sep 20, 2023
3e5898d
Merge remote-tracking branch 'origin/main' into 98_grouping_with_self…
ShiriMoran Sep 20, 2023
79ee4cd
not sure why this newline was missing; this line has to be excluded f…
ShiriMoran Sep 20, 2023
6e49140
Merge branch 'main' into 98_grouping_with_self_loops
ShiriMoran Sep 21, 2023
bfd3576
Update pkg/vpcmodel/groupingSelfLoop.go
ShiriMoran Sep 21, 2023
55b710e
Update pkg/vpcmodel/groupingSelfLoop.go
ShiriMoran Sep 21, 2023
c89f318
CR comments
ShiriMoran Sep 21, 2023
af436ba
Merge remote-tracking branch 'origin/98_grouping_with_self_loops' int…
ShiriMoran Sep 21, 2023
b9b5d97
merge with main
ShiriMoran Sep 27, 2023
767cb52
Update pkg/vpcmodel/groupingSelfLoop.go
ShiriMoran Sep 27, 2023
de4b205
Update pkg/vpcmodel/groupingSelfLoop.go
ShiriMoran Sep 27, 2023
5d0adca
Update pkg/vpcmodel/groupingSelfLoop.go
ShiriMoran Sep 27, 2023
dd59da6
simple CRs comment
ShiriMoran Sep 27, 2023
048b44e
Merge remote-tracking branch 'origin/98_grouping_with_self_loops' int…
ShiriMoran Sep 27, 2023
353fa80
Merge branch 'main' into 98_grouping_with_self_loops
ShiriMoran Sep 27, 2023
0fbd72a
CR fix - merge groups of VSI must have the same subnet in src and dst
ShiriMoran Sep 27, 2023
dc8e171
CR fix - keep unified key
ShiriMoran Sep 27, 2023
8e5a013
CR: add documentation for each function
ShiriMoran Sep 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions pkg/ibmvpc/examples/acl_testing3_with_grouping.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,7 @@ vsi1-ky[10.240.10.4] => Public Internet 161.26.0.0/16 : protocol: UDP *
vsi1-ky[10.240.10.4] => vsi2-ky[10.240.20.4] : protocol: TCP,UDP
vsi2-ky[10.240.20.4] => Public Internet (all ranges) : All Connections
vsi2-ky[10.240.20.4] => vsi1-ky[10.240.10.4] : All Connections *
vsi3a-ky[10.240.30.5],vsi3b-ky[10.240.30.6] => vsi3c-ky[10.240.30.4] : All Connections
vsi3a-ky[10.240.30.5],vsi3b-ky[10.240.30.6],vsi3c-ky[10.240.30.4] => vsi1-ky[10.240.10.4] : All Connections *
vsi3a-ky[10.240.30.5],vsi3c-ky[10.240.30.4] => vsi3b-ky[10.240.30.6] : All Connections
vsi3b-ky[10.240.30.6],vsi3c-ky[10.240.30.4] => vsi3a-ky[10.240.30.5] : All Connections
vsi3a-ky[10.240.30.5],vsi3b-ky[10.240.30.6],vsi3c-ky[10.240.30.4] => vsi3a-ky[10.240.30.5],vsi3b-ky[10.240.30.6],vsi3c-ky[10.240.30.4] : All Connections

connections are stateful unless marked with *
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
combined connections between subnets:
subnet1-ky => Public Internet (all ranges) : All Connections
subnet1-ky => subnet2-ky,subnet3-ky : All Connections
subnet2-ky => subnet1-ky,subnet3-ky : All Connections
subnet3-ky => subnet1-ky,subnet2-ky : All Connections
subnet1-ky,subnet2-ky,subnet3-ky => subnet1-ky,subnet2-ky,subnet3-ky : All Connections

connections are stateful unless marked with *
39 changes: 24 additions & 15 deletions pkg/vpcmodel/grouping.go
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,8 @@ func (g *groupingConnections) addPublicConnectivity(ep EndpointElem, conn string
(*g)[ep][conn] = append((*g)[ep][conn], targetNode)
}

// vsiGroupingBySubnets returns a slice of EndpointElem objects produced from an input slice, by grouping
// set of elements that represent network interface nodes from the same subnet into a single groupedNetworkInterfaces object
// vsiGroupingBySubnets returns a slice of EndpointElem objects, by grouping set of elements that
// represent network interface nodes from the same subnet into a single groupedNetworkInterfaces object
func vsiGroupingBySubnets(elemsList []EndpointElem, c *CloudConfig) []EndpointElem {
res := []EndpointElem{}
subnetNameToNodes := map[string][]EndpointElem{} // map from subnet name to its nodes from the input
Expand All @@ -135,7 +135,8 @@ func vsiGroupingBySubnets(elemsList []EndpointElem, c *CloudConfig) []EndpointEl
return res
}

// subnetGrouping returns a slice of EndpointElem objects produced from an input slice, by grouping EndpointElem that represents a subnet
// subnetGrouping returns a slice of EndpointElem objects produced from an input slice, by grouping
// set of elements that represent subnets into a single groupedNetworkInterfaces object
func subnetGrouping(elemsList []EndpointElem) []EndpointElem {
res := []EndpointElem{}
subnetsToGroup := []EndpointElem{} // subnets to be grouped
Expand Down Expand Up @@ -204,6 +205,20 @@ func (g *GroupConnLines) groupExternalAddressesForSubnets() {
g.GroupedLines = res
}

// aux func, returns true iff the EndpointElem is Node if grouping vsis or NodeSet if grouping subnets
func isInetrnalOfRequiredType(ep EndpointElem, groupVsi bool) bool {
ShiriMoran marked this conversation as resolved.
Show resolved Hide resolved
if groupVsi { // groups vsis Nodes
if _, ok := ep.(Node); !ok {
return false
}
} else { // groups subnets NodeSets
if _, ok := ep.(NodeSet); !ok {
return false
}
}
return true
}

// groups src/targets for either Vsis or Subnets
func (g *GroupConnLines) groupLinesByKey(srcGrouping, groupVsi bool) (res []*GroupedConnLine,
groupingSrcOrDst map[string][]*GroupedConnLine) {
Expand All @@ -213,24 +228,18 @@ func (g *GroupConnLines) groupLinesByKey(srcGrouping, groupVsi bool) (res []*Gro
// populate map groupingSrcOrDst
for _, line := range g.GroupedLines {
srcOrDst, dstOrSrc := line.getSrcOrDst(srcGrouping), line.getSrcOrDst(!srcGrouping)
if groupVsi { // groups vsis Nodes
if _, ok := srcOrDst.(Node); !ok {
res = append(res, line)
continue
}
} else { // groups subnets NodeSets
if _, ok := srcOrDst.(NodeSet); !ok {
res = append(res, line)
continue
}
if !isInetrnalOfRequiredType(srcOrDst, groupVsi) {
res = append(res, line)
continue
}
key := dstOrSrc.Name() + ";" + line.Conn
if _, ok := groupingSrcOrDst[key]; !ok {
groupingSrcOrDst[key] = []*GroupedConnLine{}
}
groupingSrcOrDst[key] = append(groupingSrcOrDst[key], line)
}
return res, groupingSrcOrDst
newGroupingSrcOrDst := g.extendGroupingSelfLoops(groupingSrcOrDst, srcGrouping)
return res, newGroupingSrcOrDst
}

// assuming the g.groupedLines was already initialized by previous step groupExternalAddresses()
Expand Down Expand Up @@ -294,7 +303,7 @@ func listEndpointElemStr(eps []EndpointElem, fn func(ep EndpointElem) string) st
endpointsStrings[i] = fn(ep)
}
sort.Strings(endpointsStrings)
return strings.Join(endpointsStrings, ",")
return strings.Join(endpointsStrings, commaSepartor)
}

func (g *groupedExternalNodes) String() string {
Expand Down
Loading