Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

124 all subnets grouping #136

Merged
merged 10 commits into from
Aug 17, 2023
Merged

124 all subnets grouping #136

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
ee600cf
So that grouping can be reused by subnetConnectivity
ShiriMoran Aug 14, 2023
43e8d41
VPCsubnetConnectivity computed and stored on EndpointElem instead of …
ShiriMoran Aug 14, 2023
3916e6a
make lint happy
ShiriMoran Aug 14, 2023
d6580bd
added grouping for analysis allSubnets
ShiriMoran Aug 15, 2023
0db151c
update tests
ShiriMoran Aug 15, 2023
5ad89a8
make lint happy
ShiriMoran Aug 15, 2023
9fe9fd7
CR comment: statefullness not computed yet
ShiriMoran Aug 15, 2023
7766b0b
lint is a real Nazi
ShiriMoran Aug 15, 2023
99c9b4d
improved a comment
ShiriMoran Aug 16, 2023
4bbfa7e
Merge branch 'main' into 124_all_subnets_grouping
ShiriMoran Aug 17, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions cmd/analyzer/parse_args.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,8 +86,8 @@ func ParseInArgs(cmdlineArgs []string) (*InArgs, error) {
return nil, fmt.Errorf("currently only txt output format supported with %s analysis type", *args.AnalysisType)
}

if *args.AnalysisType != allEndpoints && *args.Grouping {
return nil, fmt.Errorf("currently only allEndpoints analysis type supports grouping")
if *args.AnalysisType == singleSubnet && *args.Grouping {
return nil, fmt.Errorf("currently singleSubnet analysis type does not support grouping")
}

return &args, nil
Expand Down
6 changes: 3 additions & 3 deletions pkg/ibmvpc/examples/acl_testing5_oldsubnetsBased_withPGW.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
combined connections between subnets:
sub1-1-ky => Public Internet [8.8.8.8/32] : protocol: UDP dst-ports: 53
sub1-1-ky => Public Internet 8.8.8.8/32 : protocol: UDP dst-ports: 53
sub1-1-ky => sub1-2-ky : protocol: TCP
sub1-1-ky => sub1-3-ky : protocol: TCP
sub1-2-ky => sub1-1-ky : protocol: TCP
sub1-2-ky => sub1-3-ky : protocol: TCP
sub1-3-ky => sub1-1-ky : protocol: TCP
sub1-3-ky => sub1-2-ky : protocol: TCP
sub2-1-ky => Public Internet [8.8.8.8/32] : protocol: UDP dst-ports: 53
sub2-1-ky => Public Internet 8.8.8.8/32 : protocol: UDP dst-ports: 53
sub2-1-ky => sub2-2-ky : All Connections
sub2-2-ky => sub2-1-ky : All Connections
sub2-2-ky => sub2-1-ky : All Connections
6 changes: 3 additions & 3 deletions pkg/ibmvpc/examples/acl_testing5subnetsBased_withPGW.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
combined connections between subnets:
sub1-1-ky => Public Internet [8.8.8.8/32] : protocol: UDP dst-ports: 53
sub1-1-ky => Public Internet 8.8.8.8/32 : protocol: UDP dst-ports: 53
sub1-1-ky => sub1-2-ky : protocol: TCP
sub1-1-ky => sub1-3-ky : protocol: TCP
sub1-1-ky => sub3-1-ky : protocol: ICMP icmp-type: 0 icmp-code: 0
sub1-2-ky => sub1-1-ky : protocol: TCP
sub1-2-ky => sub1-3-ky : protocol: TCP
sub1-3-ky => sub1-1-ky : protocol: TCP
sub1-3-ky => sub1-2-ky : protocol: TCP
sub2-1-ky => Public Internet [8.8.8.8/32] : protocol: UDP dst-ports: 53
sub2-1-ky => Public Internet 8.8.8.8/32 : protocol: UDP dst-ports: 53
sub2-1-ky => sub2-2-ky : All Connections
sub2-1-ky => sub3-1-ky : protocol: ICMP icmp-type: 0 icmp-code: 0; protocol: TCP src-ports: 443
sub2-2-ky => sub2-1-ky : All Connections
sub3-1-ky => sub1-1-ky : protocol: ICMP icmp-type: 0 icmp-code: 0
sub3-1-ky => sub2-1-ky : protocol: ICMP icmp-type: 0 icmp-code: 0; protocol: TCP dst-ports: 443
sub3-1-ky => sub2-1-ky : protocol: ICMP icmp-type: 0 icmp-code: 0; protocol: TCP dst-ports: 443
410 changes: 4 additions & 406 deletions pkg/ibmvpc/examples/demo_with_instancessubnetsBased_withPGW.txt
View file
Open in desktop

Large diffs are not rendered by default.

163 changes: 2 additions & 161 deletions pkg/ibmvpc/examples/sg_testing1_newsubnetsBased_withPGW.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,167 +1,8 @@
combined connections between subnets:
subnet1-ky => Public Internet [1.0.0.0/8] : All Connections
subnet1-ky => Public Internet [100.0.0.0/10] : All Connections
subnet1-ky => Public Internet [100.128.0.0/9] : All Connections
subnet1-ky => Public Internet [101.0.0.0/8] : All Connections
subnet1-ky => Public Internet [102.0.0.0/7] : All Connections
subnet1-ky => Public Internet [104.0.0.0/5] : All Connections
subnet1-ky => Public Internet [11.0.0.0/8] : All Connections
subnet1-ky => Public Internet [112.0.0.0/5] : All Connections
subnet1-ky => Public Internet [12.0.0.0/6] : All Connections
subnet1-ky => Public Internet [120.0.0.0/6] : All Connections
subnet1-ky => Public Internet [124.0.0.0/7] : All Connections
subnet1-ky => Public Internet [126.0.0.0/8] : All Connections
subnet1-ky => Public Internet [128.0.0.0/5] : All Connections
subnet1-ky => Public Internet [136.0.0.0/6] : All Connections
subnet1-ky => Public Internet [140.0.0.0/7] : All Connections
subnet1-ky => Public Internet [142.0.0.0/8] : All Connections
subnet1-ky => Public Internet [143.0.0.0/8] : All Connections
subnet1-ky => Public Internet [144.0.0.0/7] : All Connections
subnet1-ky => Public Internet [146.0.0.0/8] : All Connections
subnet1-ky => Public Internet [147.0.0.0/9] : All Connections
subnet1-ky => Public Internet [147.128.0.0/10] : All Connections
subnet1-ky => Public Internet [147.192.0.0/11] : All Connections
subnet1-ky => Public Internet [147.224.0.0/13] : All Connections
subnet1-ky => Public Internet [147.232.0.0/15] : All Connections
subnet1-ky => Public Internet [147.234.0.0/16] : All Connections
subnet1-ky => Public Internet [147.235.0.0/17] : All Connections
subnet1-ky => Public Internet [147.235.128.0/18] : All Connections
subnet1-ky => Public Internet [147.235.192.0/20] : All Connections
subnet1-ky => Public Internet [147.235.208.0/21] : All Connections
subnet1-ky => Public Internet [147.235.216.0/23] : All Connections
subnet1-ky => Public Internet [147.235.218.0/24] : All Connections
subnet1-ky => Public Internet [147.235.219.0/25] : All Connections
subnet1-ky => Public Internet [147.235.219.128/26] : All Connections
subnet1-ky => Public Internet [147.235.219.192/29] : All Connections
subnet1-ky => Public Internet [147.235.219.200/30] : All Connections
subnet1-ky => Public Internet [147.235.219.204/31] : All Connections
subnet1-ky => Public Internet [147.235.219.206/32] : All Connections
subnet1-ky => Public Internet [147.235.219.207/32] : All Connections
subnet1-ky => Public Internet [147.235.219.208/28] : All Connections
subnet1-ky => Public Internet [147.235.219.224/27] : All Connections
subnet1-ky => Public Internet [147.235.220.0/22] : All Connections
subnet1-ky => Public Internet [147.235.224.0/19] : All Connections
subnet1-ky => Public Internet [147.236.0.0/14] : All Connections
subnet1-ky => Public Internet [147.240.0.0/12] : All Connections
subnet1-ky => Public Internet [148.0.0.0/6] : All Connections
subnet1-ky => Public Internet [152.0.0.0/5] : All Connections
subnet1-ky => Public Internet [16.0.0.0/4] : All Connections
subnet1-ky => Public Internet [160.0.0.0/8] : All Connections
subnet1-ky => Public Internet [161.0.0.0/12] : All Connections
subnet1-ky => Public Internet [161.128.0.0/9] : All Connections
subnet1-ky => Public Internet [161.16.0.0/13] : All Connections
subnet1-ky => Public Internet [161.24.0.0/15] : All Connections
subnet1-ky => Public Internet [161.26.0.0/16] : All Connections
subnet1-ky => Public Internet [161.27.0.0/16] : All Connections
subnet1-ky => Public Internet [161.28.0.0/14] : All Connections
subnet1-ky => Public Internet [161.32.0.0/11] : All Connections
subnet1-ky => Public Internet [161.64.0.0/10] : All Connections
subnet1-ky => Public Internet [162.0.0.0/7] : All Connections
subnet1-ky => Public Internet [164.0.0.0/6] : All Connections
subnet1-ky => Public Internet [168.0.0.0/8] : All Connections
subnet1-ky => Public Internet [169.0.0.0/9] : All Connections
subnet1-ky => Public Internet [169.128.0.0/10] : All Connections
subnet1-ky => Public Internet [169.192.0.0/11] : All Connections
subnet1-ky => Public Internet [169.224.0.0/12] : All Connections
subnet1-ky => Public Internet [169.240.0.0/13] : All Connections
subnet1-ky => Public Internet [169.248.0.0/14] : All Connections
subnet1-ky => Public Internet [169.252.0.0/15] : All Connections
subnet1-ky => Public Internet [169.255.0.0/16] : All Connections
subnet1-ky => Public Internet [170.0.0.0/7] : All Connections
subnet1-ky => Public Internet [172.0.0.0/12] : All Connections
subnet1-ky => Public Internet [172.128.0.0/9] : All Connections
subnet1-ky => Public Internet [172.32.0.0/11] : All Connections
subnet1-ky => Public Internet [172.64.0.0/10] : All Connections
subnet1-ky => Public Internet [173.0.0.0/8] : All Connections
subnet1-ky => Public Internet [174.0.0.0/7] : All Connections
subnet1-ky => Public Internet [176.0.0.0/4] : All Connections
subnet1-ky => Public Internet [192.0.1.0/24] : All Connections
subnet1-ky => Public Internet [192.0.128.0/17] : All Connections
subnet1-ky => Public Internet [192.0.16.0/20] : All Connections
subnet1-ky => Public Internet [192.0.3.0/24] : All Connections
subnet1-ky => Public Internet [192.0.32.0/19] : All Connections
subnet1-ky => Public Internet [192.0.4.0/22] : All Connections
subnet1-ky => Public Internet [192.0.64.0/18] : All Connections
subnet1-ky => Public Internet [192.0.8.0/21] : All Connections
subnet1-ky => Public Internet [192.1.0.0/16] : All Connections
subnet1-ky => Public Internet [192.128.0.0/11] : All Connections
subnet1-ky => Public Internet [192.16.0.0/12] : All Connections
subnet1-ky => Public Internet [192.160.0.0/13] : All Connections
subnet1-ky => Public Internet [192.169.0.0/16] : All Connections
subnet1-ky => Public Internet [192.170.0.0/15] : All Connections
subnet1-ky => Public Internet [192.172.0.0/14] : All Connections
subnet1-ky => Public Internet [192.176.0.0/12] : All Connections
subnet1-ky => Public Internet [192.192.0.0/10] : All Connections
subnet1-ky => Public Internet [192.2.0.0/15] : All Connections
subnet1-ky => Public Internet [192.32.0.0/11] : All Connections
subnet1-ky => Public Internet [192.4.0.0/14] : All Connections
subnet1-ky => Public Internet [192.64.0.0/12] : All Connections
subnet1-ky => Public Internet [192.8.0.0/13] : All Connections
subnet1-ky => Public Internet [192.80.0.0/13] : All Connections
subnet1-ky => Public Internet [192.88.0.0/18] : All Connections
subnet1-ky => Public Internet [192.88.100.0/22] : All Connections
subnet1-ky => Public Internet [192.88.104.0/21] : All Connections
subnet1-ky => Public Internet [192.88.112.0/20] : All Connections
subnet1-ky => Public Internet [192.88.128.0/17] : All Connections
subnet1-ky => Public Internet [192.88.64.0/19] : All Connections
subnet1-ky => Public Internet [192.88.96.0/23] : All Connections
subnet1-ky => Public Internet [192.88.98.0/24] : All Connections
subnet1-ky => Public Internet [192.89.0.0/16] : All Connections
subnet1-ky => Public Internet [192.90.0.0/15] : All Connections
subnet1-ky => Public Internet [192.92.0.0/14] : All Connections
subnet1-ky => Public Internet [192.96.0.0/11] : All Connections
subnet1-ky => Public Internet [193.0.0.0/8] : All Connections
subnet1-ky => Public Internet [194.0.0.0/7] : All Connections
subnet1-ky => Public Internet [196.0.0.0/7] : All Connections
subnet1-ky => Public Internet [198.0.0.0/12] : All Connections
subnet1-ky => Public Internet [198.128.0.0/9] : All Connections
subnet1-ky => Public Internet [198.16.0.0/15] : All Connections
subnet1-ky => Public Internet [198.20.0.0/14] : All Connections
subnet1-ky => Public Internet [198.24.0.0/13] : All Connections
subnet1-ky => Public Internet [198.32.0.0/12] : All Connections
subnet1-ky => Public Internet [198.48.0.0/15] : All Connections
subnet1-ky => Public Internet [198.50.0.0/16] : All Connections
subnet1-ky => Public Internet [198.51.0.0/18] : All Connections
subnet1-ky => Public Internet [198.51.101.0/24] : All Connections
subnet1-ky => Public Internet [198.51.102.0/23] : All Connections
subnet1-ky => Public Internet [198.51.104.0/21] : All Connections
subnet1-ky => Public Internet [198.51.112.0/20] : All Connections
subnet1-ky => Public Internet [198.51.128.0/17] : All Connections
subnet1-ky => Public Internet [198.51.64.0/19] : All Connections
subnet1-ky => Public Internet [198.51.96.0/22] : All Connections
subnet1-ky => Public Internet [198.52.0.0/14] : All Connections
subnet1-ky => Public Internet [198.56.0.0/13] : All Connections
subnet1-ky => Public Internet [198.64.0.0/10] : All Connections
subnet1-ky => Public Internet [199.0.0.0/8] : All Connections
subnet1-ky => Public Internet [2.0.0.0/7] : All Connections
subnet1-ky => Public Internet [200.0.0.0/7] : All Connections
subnet1-ky => Public Internet [202.0.0.0/8] : All Connections
subnet1-ky => Public Internet [203.0.0.0/18] : All Connections
subnet1-ky => Public Internet [203.0.112.0/24] : All Connections
subnet1-ky => Public Internet [203.0.114.0/23] : All Connections
subnet1-ky => Public Internet [203.0.116.0/22] : All Connections
subnet1-ky => Public Internet [203.0.120.0/21] : All Connections
subnet1-ky => Public Internet [203.0.128.0/17] : All Connections
subnet1-ky => Public Internet [203.0.64.0/19] : All Connections
subnet1-ky => Public Internet [203.0.96.0/20] : All Connections
subnet1-ky => Public Internet [203.1.0.0/16] : All Connections
subnet1-ky => Public Internet [203.128.0.0/9] : All Connections
subnet1-ky => Public Internet [203.16.0.0/12] : All Connections
subnet1-ky => Public Internet [203.2.0.0/15] : All Connections
subnet1-ky => Public Internet [203.32.0.0/11] : All Connections
subnet1-ky => Public Internet [203.4.0.0/14] : All Connections
subnet1-ky => Public Internet [203.64.0.0/10] : All Connections
subnet1-ky => Public Internet [203.8.0.0/13] : All Connections
subnet1-ky => Public Internet [204.0.0.0/6] : All Connections
subnet1-ky => Public Internet [208.0.0.0/4] : All Connections
subnet1-ky => Public Internet [32.0.0.0/3] : All Connections
subnet1-ky => Public Internet [4.0.0.0/6] : All Connections
subnet1-ky => Public Internet [64.0.0.0/3] : All Connections
subnet1-ky => Public Internet [8.0.0.0/7] : All Connections
subnet1-ky => Public Internet [96.0.0.0/6] : All Connections
subnet1-ky => Public Internet (all ranges) : All Connections
subnet1-ky => subnet2-ky : All Connections
subnet1-ky => subnet3-ky : All Connections
subnet2-ky => subnet1-ky : All Connections
subnet2-ky => subnet3-ky : All Connections
subnet3-ky => subnet1-ky : All Connections
subnet3-ky => subnet2-ky : All Connections
subnet3-ky => subnet2-ky : All Connections
6 changes: 3 additions & 3 deletions pkg/ibmvpc/vpc.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -608,11 +608,11 @@ func (pgw *PublicGateway) ConnectivityMap() map[string]vpcmodel.ConfigBasedConne
res := map[string]vpcmodel.ConfigBasedConnectivityResults{}
for _, subnetCidr := range pgw.subnetCidr {
res[subnetCidr] = vpcmodel.ConfigBasedConnectivityResults{
IngressAllowedConns: map[string]*common.ConnectionSet{},
EgressAllowedConns: map[string]*common.ConnectionSet{},
IngressAllowedConns: map[vpcmodel.EndpointElem]*common.ConnectionSet{},
EgressAllowedConns: map[vpcmodel.EndpointElem]*common.ConnectionSet{},
}
for _, dst := range pgw.destinations {
res[subnetCidr].EgressAllowedConns[dst.Name()] = vpcmodel.AllConns()
res[subnetCidr].EgressAllowedConns[dst] = vpcmodel.AllConns()
}
}

Expand Down
61 changes: 53 additions & 8 deletions pkg/vpcmodel/grouping.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ import (

const commaSepartor = ","

type groupingConnections map[Node]map[string][]Node // for each line here can group list of external nodes to cidrs list as of one element
// for each line here can group list of external nodes to cidrs list as of one element
type groupingConnections map[EndpointElem]map[string][]Node

func (g *groupingConnections) getGroupedConnLines(isSrcToDst bool) []*GroupedConnLine {
res := []*GroupedConnLine{}
Expand All @@ -29,7 +30,7 @@ func (g *groupingConnections) getGroupedConnLines(isSrcToDst bool) []*GroupedCon
}

func newGroupingConnections() *groupingConnections {
res := groupingConnections(map[Node]map[string][]Node{})
res := groupingConnections(map[EndpointElem]map[string][]Node{})
return &res
}

Expand All @@ -39,9 +40,18 @@ func newGroupConnLines(c *CloudConfig, v *VPCConnectivity, grouping bool) *Group
return res
}

func newGroupConnLinesSubnetConnectivity(c *CloudConfig, s *VPCsubnetConnectivity) *GroupConnLines {
res := &GroupConnLines{c: c, s: s, srcToDst: newGroupingConnections(), dstToSrc: newGroupingConnections()}
res.groupExternalAddressesForSubnets()
return res
}

// GroupConnLines used both for VPCConnectivity and for VPCsubnetConnectivity, one at a time. The other must be nil
// todo: define abstraction above both?
type GroupConnLines struct {
c *CloudConfig
v *VPCConnectivity
s *VPCsubnetConnectivity
srcToDst *groupingConnections
dstToSrc *groupingConnections
GroupedLines []*GroupedConnLine
Expand Down Expand Up @@ -87,14 +97,14 @@ func (g *groupedExternalNodes) Name() string {
return prefix + g.String()
}

func (g *groupingConnections) addPublicConnectivity(n Node, conn string, target Node) {
if _, ok := (*g)[n]; !ok {
(*g)[n] = map[string][]Node{}
func (g *groupingConnections) addPublicConnectivity(ep EndpointElem, conn string, targetNode Node) {
if _, ok := (*g)[ep]; !ok {
(*g)[ep] = map[string][]Node{}
}
if _, ok := (*g)[n][conn]; !ok {
(*g)[n][conn] = []Node{}
if _, ok := (*g)[ep][conn]; !ok {
(*g)[ep][conn] = []Node{}
}
(*g)[n][conn] = append((*g)[n][conn], target)
(*g)[ep][conn] = append((*g)[ep][conn], targetNode)
}

// subnetGrouping returns a slice of EndpointElem objects produced from an input slice, by grouping
Expand Down Expand Up @@ -150,6 +160,29 @@ func (g *GroupConnLines) groupExternalAddresses() {
g.GroupedLines = res
}

func (g *GroupConnLines) groupExternalAddressesForSubnets() {
// groups public internet ranges in dst when dst is public internet
res := []*GroupedConnLine{}
for src, endpointConns := range g.s.AllowedConnsCombined {
for dst, conns := range endpointConns {
if conns.IsEmpty() {
continue
}
connString := conns.EnhancedString()
if dstNode, ok := dst.(Node); ok && dstNode.IsPublicInternet() {
g.srcToDst.addPublicConnectivity(src, connString, dstNode)
} else { // since pgw enable only egress src can not be public internet, the above is the only option of public internet
// not an external connection in source or destination - nothing to group, just append
res = append(res, &GroupedConnLine{src, dst, connString})
}
}
}
// add to res lines from srcToDst and DstToSrc groupings
res = append(res, g.srcToDst.getGroupedConnLines(true)...)
res = append(res, g.dstToSrc.getGroupedConnLines(false)...)
g.GroupedLines = res
}

// assuming the g.groupedLines was already initialized by previous step groupExternalAddresses()
func (g *GroupConnLines) groupSubnetsSrcOrDst(srcGrouping bool) {
res := []*GroupedConnLine{}
Expand Down Expand Up @@ -209,6 +242,18 @@ func (g *GroupConnLines) String() string {
return strings.Join(linesStr, "\n") + asteriskDetails
}

// StringTmpWA ToDo: tmp WA until https://github.com/np-guard/vpc-network-config-analyzer/issues/138.
//
// Once the issue is solved this code can be deleted
func (g *GroupConnLines) StringTmpWA() string {
linesStr := make([]string, len(g.GroupedLines))
for i, line := range g.GroupedLines {
linesStr[i] = line.String()
}
sort.Strings(linesStr)
return strings.Join(linesStr, "\n")
}

func listNodesStr(nodes []Node, fn func(Node) string) string {
nodesStrings := make([]string, len(nodes))
for i, n := range nodes {
Expand Down
Loading