Internal Error: Received RST_STREAM with error code 2 #156
Comments
|
I have the exact same issue in my environment. I only added the These are the roles assigned to the Service Account attached to the VM that runs Forseti: forseti_identity_org_roles = [
"roles/appengine.appViewer",
"roles/bigquery.metadataViewer",
"roles/browser",
"roles/cloudasset.viewer",
"roles/cloudsql.viewer",
"roles/compute.networkViewer",
"roles/iam.securityReviewer",
"roles/orgpolicy.policyViewer",
"roles/servicemanagement.quotaViewer",
"roles/serviceusage.serviceUsageConsumer"
]
forseti_identity_project_roles = [
"roles/cloudsql.client",
"roles/logging.logWriter",
"roles/monitoring.metricWriter",
"roles/storage.objectViewer",
"roles/storage.objectCreator"
]This is my terraform configuration for Forseti: module "forseti" {
source = "terraform-google-modules/forseti/google"
version = "~> 5.2.2"
gsuite_admin_email = "[email protected]"
domain = "XXXYYYZZZ"
project_id = module.forseti_project.project_id
org_id = local.organization_id
composite_root_resources = [
"organizations/${local.organization_id}"
]
server_private = true
client_enabled = false
cloudsql_private = true
server_region = "europe-west2"
cloudsql_region = "europe-west2"
storage_bucket_location = "europe-west2"
bucket_cai_location = "europe-west2"
network = google_compute_network.forseti_network.name
subnetwork = google_compute_subnetwork.forseti_subnetwork.id
server_service_account = google_service_account.forseti_identity.email
cscc_violations_enabled = true
cscc_source_id = local.forseti_source_id
server_type = "n1-standard-4"
admin_disable_polling = true
server_grpc_allow_ranges = []
# Scanners
enabled_apis_enabled = false
blacklist_enabled = false
bigquery_enabled = false
bucket_acl_enabled = false
cloudsql_acl_enabled = false
audit_logging_enabled = false
firewall_rule_enabled = false
forwarding_rule_enabled = false
group_enabled = false
groups_settings_enabled = false
iam_policy_enabled = false
iap_enabled = false
instance_network_interface_enabled = false
ke_scanner_enabled = false
ke_version_scanner_enabled = false
kms_scanner_enabled = false
lien_enabled = false
location_enabled = false
log_sink_enabled = false
resource_enabled = false
role_enabled = false
service_account_key_enabled = false
cloudbilling_disable_polling = true
compute_disable_polling = true
container_disable_polling = true
crm_disable_polling = true
groups_settings_disable_polling = true
iam_disable_polling = true
logging_disable_polling = true
servicemanagement_disable_polling = true
serviceusage_disable_polling = true
sqladmin_disable_polling = true
appengine_disable_polling = true
bigquery_disable_polling = true
storage_disable_polling = true
config_validator_enabled = true
config_validator_violations_should_notify = true
}I've stepped through the |
|
Ok, that was quick. I solved it with the help of this bug. I've removed things like What puzzles me is that I have another organization where this worked absolutely fine a few months ago (Forseti TF module v5.2.0). |
|
Thanks @debakkerb I made this change, which fixed the error...but the constraint is not being executed now. Are you facing a similar issue? |
|
Unfortunately not, I was a bit too trigger happy I'm afraid. Btw, you can also replace However, that didn't fire off any policy violations, even though I added the |
|
Did you guys found any workaround ?
No violations found :/ nb: i was wondering why rules in contrainst folder have a field |
|
@krab-skunk or @debakkerb was there an older version of Forseti that worked for you? Did this happen after an upgrade? |
|
@onetwopunch It never worked for me, i'm fairly new to forseti (1 month), so i was focusing on trying out the python scanners ( that work perfectly fine ). For both versions (2.23.2 and 2.25.2), i was installing them from scratch using terraform. If someone could tell me which combination of forseti release + config validator commit works great, i'd be happy to try it out , there is a 7k+ppl IT company that everyone know very well i'm sure counting on it as we want to get rid of redlock ;P |
|
Worth adding, that yaml rules that does not contain organizations/..... things inside like this one iam_audit_log_all.yaml Works like a charm, just got 14000 notifications |
|
@onetwopunch @debakkerb The trick is to use forseti from master branch and along with config validator running on docker ;) Here is my terraform file And this is how i run config validator: Thanks again @gkowalski-google for the tip to try using master branch instead :) |
|
@krab-skunk I'm having the exact issue right now. Can you reply with the image version you're using? I'm using the following off of the charts I have a single constraint I'm trying to test, without any wildcards, as per the previous comments: Both are ancient images, but I've only had bad luck trying newer ones. If you can point me to something that will get the validator to not break ("RST_STREAM with error code 2") and also detect constraint violations you'll make my weekend much much better 👍 Edit: this is the scanner output I get from the orchestrator: 10.118.253.219:50052 is the IP for the config-validator service I'm using |
|
I'm running the latest, basically all i do on forseti server is runing this docker, that pull latest image And with this image, u don't have to care about - "organizations/myorgid", you can leave it like - "organizations/** as a starter ,you can try this very simple policy: restrict_fw_rules_world_open.yaml |
|
Ah, so you're running forseti on a VM and simply running the validator in a container? What version of forseti are you using? My problem is that the orchestrator, config validator and the server run in containers. So i'm struggling to figure out which version of server/config-validator work well with each other. At this point I either get the RST_STREAM error or simply no violations are recorded. |
|
Yes, the GKE version is some kind of alpha stuff, and i plan to put that in Prod with hundreds of GKE clusters that i wanna monitor, so for now, as simple POC, i try not to go too too wild. So i went the VM way for now, forseti on a VM , and config validator on a container in this same VM. As you can see in my terraform file, i run the latest forseti from master branch, coz its the only one that worked for me as you can read in my posts above ;) |
|
Yeah, so I just tried again with the following images:
I can see that the validator is properly copying the constraint I want to test from GCS: There's no errors thrown anywhere, however no violations are detected by the config_validator: I couldnt help but notice this message, but I doubt its an issue: I guess I won't end my week with a win. Thanks anyway. |
|
Well, for sure some others here are way more expert than me in this matter, but if you set up composite_root in your forseti config server file, well, i think its look pretty clear that config validator scanner will not work with it :/ Personally i'm not using it, i'm using instead
along with lots of excluded_resources: [...] |
|
Yeah I switched from it to root and retried. The reason for using the composite root is because I can't use a folder in the root (and an inventory for my whole org would take too long). Based on the docs, something like Regardless, I can now see config_validator recording violations: However nothing is recorded by the notifier output to GCS The content of scanner_violations only shows the enabled_apis rule violations, but not the two config_validator_violations that were reported. Seeing 2 config_validator_violations captured is progress :) - However nothing is captured in the output (nor sent to Security Command Center) Edit: I think I see the problem 👍 !!!! |
Totally understand that, that's why i put plenty of excluded folders and projects for my POC, Make sure when you run the notifier, that the ID you use is indeed the one returned by your scanner forseti notifier run --scanner_index_id SCANNER_ID and also that the source_id for your violation/cscc is well configured ;) Perso, each time i run a test, i launch the following command in order: Good luck |
|
If you are using the terraform module to install Forseti with CV, make sure your policy files use the singular form of I was facing this issue with |
I've just migrated to use Config Validator using Terraform. I have a few custom Rego files, all of which pass tests locally and are fairly simple. The config-validator service is running just fine on the server (i.e.
sudo systemctl status config-validator). When I run forseti using the command in the crontab I get a 500 error:Received RST_STREAM with error code 2, which, I'm assuming, is why none of my config validator constraints are being executed. I've tried this multiple times with the same error so it's not an ephemeral error. At the time of running it https://status.cloud.google.com/ is all green too so I'm not sure what's going on.Steps to reproduce (from the forseti-server)
Full Error from Cloud Logging
The text was updated successfully, but these errors were encountered: