Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance Error handling #163

Merged
merged 38 commits into from
Feb 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
59468fd
Implement warning for missing output
adhilto Jan 23, 2024
d2c5fe6
Add missing reo spf check
adhilto Jan 24, 2024
74ce767
Add rego check for 10.2
adhilto Jan 24, 2024
33ad897
Add rego check for 15.2
adhilto Jan 24, 2024
09d6e05
Add rego check for 16.2
adhilto Jan 24, 2024
9156aad
Add missing rego checks for group 18
adhilto Jan 24, 2024
e358c87
Broke up the run_reporter function to satisfy linter
adhilto Jan 24, 2024
c2770e6
Initial commit of error handling code
adhilto Jan 25, 2024
4b0641e
Merge latest changes from main
adhilto Jan 25, 2024
8e83448
merge latest changes from main
adhilto Jan 25, 2024
0e1b301
Fix bad merge
adhilto Jan 25, 2024
75e8b8a
Simplify Tenant Metadata (#152)
adhilto Jan 25, 2024
7228bad
Merge latest changes from #158, Ensure Each Control Is Included in the
adhilto Jan 25, 2024
e071a83
Fix bad merge
adhilto Jan 25, 2024
940d2cf
Resolve conflicts AGAIN
adhilto Jan 25, 2024
4b3573b
Correct gmail bug, forgot to preface the dns function calls with self
adhilto Jan 25, 2024
a3dcb3a
Refactor long lines
adhilto Jan 25, 2024
3a29cbe
Broke out error handling code into separate functions
adhilto Jan 25, 2024
f5514b8
correct error with f string
adhilto Jan 25, 2024
7fc84ca
Add special case prereqs to rego files
adhilto Jan 25, 2024
a6564e2
Modify groups prereqs
adhilto Jan 25, 2024
f9f8280
Correct typo in reports API reference
adhilto Jan 25, 2024
cfdc934
If prereqs not defined, assume dependence on reports api
adhilto Jan 25, 2024
808b6e4
Merge branch 'main' into error-handling
adhilto Jan 30, 2024
22832e5
Fix error in provider from merge
adhilto Jan 30, 2024
88a752c
Add missing tests for gws 18
adhilto Jan 30, 2024
20842ff
Fix other regressions caused by merge
adhilto Jan 30, 2024
6eff1b8
Make the linter happy
adhilto Jan 30, 2024
d553e88
Refactor long lines
adhilto Jan 30, 2024
5a301ae
Fix groups API reference links
adhilto Jan 30, 2024
fc3f476
Refactor services object to a state variable
adhilto Feb 1, 2024
918400e
Make customer_id and dnsclient state variables
adhilto Feb 1, 2024
0774d30
Ensure the domains() API is called only once
adhilto Feb 1, 2024
d5300aa
Changes to satisfy the linter
adhilto Feb 1, 2024
00c959c
Convert API reference to enums
adhilto Feb 1, 2024
29f4fcd
Correct URL bugs
adhilto Feb 1, 2024
2cce5fc
Add documentation
adhilto Feb 1, 2024
1273cab
Rename api_reference to types
adhilto Feb 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 26 additions & 4 deletions Testing/RegoTests/gmail/gmail03_test.rego
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,31 @@ import future.keywords
#
# GWS.GMAIL.3.1v0.1
#--
test_MaintainList_Correct_V1 if {
# Test not implemented
PolicyId := "GWS.GMAIL.17.1v0.1"
Output := tests with input as {
"gmail_logs": {"items": [
]},
"tenant_info": {
"topLevelOU": ""
}
}

RuleOutput := [Result | some Result in Output; Result.PolicyId == PolicyId]
count(RuleOutput) == 1
not RuleOutput[0].RequirementMet
not RuleOutput[0].NoSuchEvent
RuleOutput[0].ReportDetails == "Currently not able to be tested automatically; please manually check."
}
#--

#
# GWS.GMAIL.3.2v0.1
#--
test_SPF_Correct_V1 if {
# Test SPF when there's only one domain
PolicyId := "GWS.GMAIL.3.1v0.1"
PolicyId := "GWS.GMAIL.3.2v0.1"
Output := tests with input as {
"spf_records": [
{
Expand All @@ -27,7 +49,7 @@ test_SPF_Correct_V1 if {

test_SPF_Correct_V2 if {
# Test SPF when there's multiple domains
PolicyId := "GWS.GMAIL.3.1v0.1"
PolicyId := "GWS.GMAIL.3.2v0.1"
Output := tests with input as {
"spf_records": [
{
Expand All @@ -51,7 +73,7 @@ test_SPF_Correct_V2 if {

test_SPF_Incorrect_V1 if {
# Test SPF when there's multiple domains and only one is correct
PolicyId := "GWS.GMAIL.3.1v0.1"
PolicyId := "GWS.GMAIL.3.2v0.1"
Output := tests with input as {
"spf_records": [
{
Expand All @@ -75,7 +97,7 @@ test_SPF_Incorrect_V1 if {

test_SPF_Incorrect_V2 if {
# Test SPF when there's only one domain and it's wrong
PolicyId := "GWS.GMAIL.3.1v0.1"
PolicyId := "GWS.GMAIL.3.2v0.1"
Output := tests with input as {
"spf_records": [
{
Expand Down
21 changes: 21 additions & 0 deletions Testing/RegoTests/gmail/gmail10_test.rego
Original file line number Diff line number Diff line change
Expand Up @@ -310,5 +310,26 @@ test_GoogleWorkspaceSync_Incorrect_V5 if {
not RuleOutput[0].NoSuchEvent
RuleOutput[0].ReportDetails == "Requirement failed in Secondary OU."
}
#--

#
# GWS.GMAIL.10.2v0.1
#--
test_May_Correct_V1 if {
# Test not implemented
PolicyId := "GWS.GMAIL.10.2v0.1"
Output := tests with input as {
"gmail_logs": {"items": [
]},
"tenant_info": {
"topLevelOU": ""
}
}

RuleOutput := [Result | some Result in Output; Result.PolicyId == PolicyId]
count(RuleOutput) == 1
not RuleOutput[0].RequirementMet
not RuleOutput[0].NoSuchEvent
RuleOutput[0].ReportDetails == "Currently not able to be tested automatically; please manually check."
}
#--
22 changes: 22 additions & 0 deletions Testing/RegoTests/gmail/gmail15_test.rego
Original file line number Diff line number Diff line change
Expand Up @@ -348,4 +348,26 @@ test_EnhancedPreDeliveryMessageScanning_Incorrect_V5 if {
not RuleOutput[0].NoSuchEvent
RuleOutput[0].ReportDetails == "Requirement failed in Secondary OU."
}
#--

#
# GWS.GMAIL.15.2v0.1
#--
test_Other_Correct_V1 if {
# Test not implemented
PolicyId := "GWS.GMAIL.15.2v0.1"
Output := tests with input as {
"gmail_logs": {"items": [
]},
"tenant_info": {
"topLevelOU": ""
}
}

RuleOutput := [Result | some Result in Output; Result.PolicyId == PolicyId]
count(RuleOutput) == 1
not RuleOutput[0].RequirementMet
not RuleOutput[0].NoSuchEvent
RuleOutput[0].ReportDetails == "Currently not able to be tested automatically; please manually check."
}
#--
22 changes: 22 additions & 0 deletions Testing/RegoTests/gmail/gmail16_test.rego
Original file line number Diff line number Diff line change
Expand Up @@ -310,4 +310,26 @@ test_SecuritySandbox_Incorrect_V5 if {
not RuleOutput[0].NoSuchEvent
RuleOutput[0].ReportDetails == "Requirement failed in Secondary OU."
}
#--

#
# GWS.GMAIL.16.2v0.1
#--
test_Other_Correct_V1 if {
# Test not implemented
PolicyId := "GWS.GMAIL.16.2v0.1"
Output := tests with input as {
"gmail_logs": {"items": [
]},
"tenant_info": {
"topLevelOU": ""
}
}

RuleOutput := [Result | some Result in Output; Result.PolicyId == PolicyId]
count(RuleOutput) == 1
not RuleOutput[0].RequirementMet
not RuleOutput[0].NoSuchEvent
RuleOutput[0].ReportDetails == "Currently not able to be tested automatically; please manually check."
}
#--
44 changes: 44 additions & 0 deletions Testing/RegoTests/gmail/gmail18_test.rego
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,48 @@ test_AdvanvedEmailContentFitlering_Correct_V1 if {
not RuleOutput[0].NoSuchEvent
RuleOutput[0].ReportDetails == "Currently not able to be tested automatically; please manually check."
}
#--

#
# GWS.GMAIL.18.2v0.1
#--
test_Other_Correct_V1 if {
# Test not implemented
PolicyId := "GWS.GMAIL.18.2v0.1"
Output := tests with input as {
"gmail_logs": {"items": [
]},
"tenant_info": {
"topLevelOU": ""
}
}

RuleOutput := [Result | some Result in Output; Result.PolicyId == PolicyId]
count(RuleOutput) == 1
not RuleOutput[0].RequirementMet
not RuleOutput[0].NoSuchEvent
RuleOutput[0].ReportDetails == "Currently not able to be tested automatically; please manually check."
}
#--

#
# GWS.GMAIL.18.3v0.1
#--
test_PII_Correct_V1 if {
# Test not implemented
PolicyId := "GWS.GMAIL.18.3v0.1"
Output := tests with input as {
"gmail_logs": {"items": [
]},
"tenant_info": {
"topLevelOU": ""
}
}

RuleOutput := [Result | some Result in Output; Result.PolicyId == PolicyId]
count(RuleOutput) == 1
not RuleOutput[0].RequirementMet
not RuleOutput[0].NoSuchEvent
RuleOutput[0].ReportDetails == "Currently not able to be tested automatically; please manually check."
}
#--
1 change: 1 addition & 0 deletions rego/Commoncontrols.rego
Original file line number Diff line number Diff line change
Expand Up @@ -713,6 +713,7 @@ tests contains {
#--
tests contains {
"PolicyId": "GWS.COMMONCONTROLS.7.1v0.1",
"Prerequisites": ["directory/v1/users/list"],
LaurenBassett marked this conversation as resolved.
Show resolved Hide resolved
"Criticality": "Shall",
"ReportDetails": concat("", [
concat("", ["The following super admins are configured: ", concat(", ", SuperAdmins)]),
Expand Down
94 changes: 92 additions & 2 deletions rego/Gmail.rego
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ DomainsWithDkim contains DkimRecord.domain if {

tests contains {
"PolicyId": "GWS.GMAIL.2.1v0.1",
"Prerequisites": ["directory/v1/domains/list", "get_dkim_records"],
"Criticality": "Should",
"ReportDetails": ReportDetailsArray(Status, DomainsWithoutDkim, AllDomains),
"ActualValue": input.dkim_records,
Expand All @@ -105,14 +106,29 @@ if {
#
# Baseline GWS.GMAIL.3.1v0.1
#--
# No implementation steps provided for this policy
tests contains {
"PolicyId": "GWS.GMAIL.3.1v0.1",
"Criticality": "Shall/Not-Implemented",
"ReportDetails": "Currently not able to be tested automatically; please manually check.",
"ActualValue": "",
"RequirementMet": false,
"NoSuchEvent": false
}
#--

#
# Baseline GWS.GMAIL.3.2v0.1
#--
DomainsWithSpf contains SpfRecord.domain if {
some SpfRecord in input.spf_records
some Rdata in SpfRecord.rdata
startswith(Rdata, "v=spf1 ")
}

tests contains {
"PolicyId": "GWS.GMAIL.3.1v0.1",
"PolicyId": "GWS.GMAIL.3.2v0.1",
"Prerequisites": ["directory/v1/domains/list", "get_spf_records"],
"Criticality": "Shall",
"ReportDetails": ReportDetailsArray(Status, DomainsWithoutSpf, AllDomains),
"ActualValue": DomainsWithoutSpf,
Expand Down Expand Up @@ -141,6 +157,7 @@ DomainsWithDmarc contains DmarcRecord.domain if {

tests contains {
"PolicyId": "GWS.GMAIL.4.1v0.1",
"Prerequisites": ["directory/v1/domains/list", "get_dmarc_records"],
"Criticality": "Shall",
"ReportDetails": ReportDetailsArray(Status, DomainsWithoutDmarc, AllDomains),
"ActualValue": input.dmarc_records,
Expand All @@ -164,6 +181,7 @@ DomainsWithPreject contains DmarcRecord.domain if {

tests contains {
"PolicyId": "GWS.GMAIL.4.2v0.1",
"Prerequisites": ["directory/v1/domains/list", "get_dmarc_records"],
"Criticality": "Shall",
"ReportDetails": ReportDetailsArray(Status, DomainsWithoutPreject, AllDomains),
"ActualValue": input.dmarc_records,
Expand All @@ -187,6 +205,7 @@ DomainsWithDHSContact contains DmarcRecord.domain if {

tests contains {
"PolicyId": "GWS.GMAIL.4.3v0.1",
"Prerequisites": ["directory/v1/domains/list", "get_dmarc_records"],
"Criticality": "Shall",
"ReportDetails": ReportDetailsArray(Status, DomainsWithoutDHSContact, AllDomains),
"ActualValue": input.dmarc_records,
Expand All @@ -210,6 +229,7 @@ DomainsWithAgencyContact contains DmarcRecord.domain if {

tests contains {
"PolicyId": "GWS.GMAIL.4.4v0.1",
"Prerequisites": ["directory/v1/domains/list", "get_dmarc_records"],
"Criticality": "Should",
"ReportDetails": ReportDetailsArray(Status, DomainsWithoutAgencyContact, AllDomains),
"ActualValue": input.dmarc_records,
Expand Down Expand Up @@ -515,7 +535,8 @@ tests contains {
"ReportDetails": "Currently not able to be tested automatically; please manually check.",
"ActualValue": "",
"RequirementMet": false,
"NoSuchEvent": false}
"NoSuchEvent": false
}
#--

###############
Expand Down Expand Up @@ -1347,6 +1368,19 @@ if {
}
#--

#
# Baseline GWS.GMAIL.10.2v0.1
#--
# No implementation steps provided for this policy
tests contains {
"PolicyId": "GWS.GMAIL.10.2v0.1",
"Criticality": "May/Not-Implemented",
"ReportDetails": "Currently not able to be tested automatically; please manually check.",
"ActualValue": "",
"RequirementMet": false,
"NoSuchEvent": false
}
#--

################
# GWS.GMAIL.11 #
Expand Down Expand Up @@ -1606,6 +1640,19 @@ if {
}
#--

#
# Baseline GWS.GMAIL.15.2v0.1
#--
# No implementation steps provided for this policy
tests contains {
"PolicyId": "GWS.GMAIL.15.2v0.1",
"Criticality": "Should/Not-Implemented",
"ReportDetails": "Currently not able to be tested automatically; please manually check.",
"ActualValue": "",
"RequirementMet": false,
"NoSuchEvent": false
}
#--

################
# GWS.GMAIL.16 #
Expand Down Expand Up @@ -1658,6 +1705,19 @@ if {
}
#--

#
# Baseline GWS.GMAIL.16.2v0.1
#--
# No implementation steps provided for this policy
tests contains {
"PolicyId": "GWS.GMAIL.16.2v0.1",
"Criticality": "Should/Not-Implemented",
"ReportDetails": "Currently not able to be tested automatically; please manually check.",
"ActualValue": "",
"RequirementMet": false,
"NoSuchEvent": false
}
#--

################
# GWS.GMAIL.17 #
Expand Down Expand Up @@ -1696,4 +1756,34 @@ tests contains {
"RequirementMet": false,
"NoSuchEvent": false
}
#--

#
# Baseline GWS.GMAIL.18.2v0.1
#--
# At this time we are unable to test because settings are configured in the GWS Admin Console
# and not available within the generated logs
tests contains {
"PolicyId": "GWS.GMAIL.18.2v0.1",
"Criticality": "Should/Not-Implemented",
"ReportDetails": "Currently not able to be tested automatically; please manually check.",
"ActualValue": "",
"RequirementMet": false,
"NoSuchEvent": false
}
#--

#
# Baseline GWS.GMAIL.18.3v0.1
#--
# At this time we are unable to test because settings are configured in the GWS Admin Console
# and not available within the generated logs
tests contains {
"PolicyId": "GWS.GMAIL.18.3v0.1",
"Criticality": "Shall/Not-Implemented",
"ReportDetails": "Currently not able to be tested automatically; please manually check.",
"ActualValue": "",
"RequirementMet": false,
"NoSuchEvent": false
}
#--
2 changes: 2 additions & 0 deletions rego/Groups.rego
Original file line number Diff line number Diff line change
Expand Up @@ -356,6 +356,7 @@ NonCompliantGroups7_1 contains Group.name if {
# if there are no groups, it has to be safe.
tests contains {
"PolicyId": "GWS.GROUPS.7.1v0.1",
"Prerequisites": ["directory/v1/domains/list", "directory/v1/groups/list", "groups-settings/v1/groups/get"],
"Criticality": "Should",
"ReportDetails": NoGroupsDetails(Groups),
"ActualValue": NoGroupsDetails(Groups),
Expand All @@ -370,6 +371,7 @@ if {
# if there are groups
tests contains {
"PolicyId": "GWS.GROUPS.7.1v0.1",
"Prerequisites": ["directory/v1/domains/list", "directory/v1/groups/list", "groups-settings/v1/groups/get"],
"Criticality": "Should",
"ReportDetails": ReportDetailsGroups(NonCompliantGroups7_1),
"ActualValue": {"NonCompliantGroups": NonCompliantGroups7_1},
Expand Down
Loading
Loading