Implement persistent findings storage - Part 2 #493

andream16 · 2024-12-05T18:04:56Z

Will open it once I refactor some things based on changes in schema.

Part 2 of #492.

How to review these changes

Review this PR
Review Implement persistent findings storage - Part 3 #494
Review Implement persistent findings storage - Part 4 #496
Review https://github.com/smithy-security/saas/pull/806

Generating protos for OCSF + Findings Service (GRPC Gateway)
Finalising schema for findings
Adding richer VulnerabilityFinding helper struct
Implementing and testing GRPC client storage backend for components
Implementing PSQL store for local docker setup + migrations
Updating docs

Q/A

Why do we have a copy of OCSF proto schema? Because we can't reference remote protobuf messages in local protobufs currently. We can explore this with buf remote registries in future.
Why SQLC? Simple way to get generated code safe interactions and models with DB with the chosen driver without hidden logic or magic. Quite good and simple for the SDK. It figures out the model based on provided schemas in migrations.
Why Atlas? We use atlas in SAAS for migrations already so this is more consistent.

proto/ocsf/ocsf_ext/finding_info/v1/finding_info.proto

proto/remote_store/findings_service/v1/findings_service.proto

Makefile

buf.gen.yaml

sdk/component/store/remote/postgresql/sqlc/query.sql

sdk/tools.go

sdk/component/conf.go

sdk/README.md

sdk/Makefile

sdk/component/store/remote/postgresql/store.go

sdk/component/vulnerability-finding/finding.go

sdk/component/component.go

sdk/component/store/remote/findings-client/client_test.go

sdk/component/conf.go

sdk/component/store/remote/postgresql/README.md

dlicheva

Looks good to me! I've only added some optional comments.

sdk/component/store/remote/findings-client/client.go

sdk/component/store/remote/postgresql/store_test.go

sdk/Makefile

sdk/component/store/local/store.go

sdk/component/store/remote/postgresql/sqlc/migrations/20241209103854_findings_table.sql

…ty type and storages. # Conflicts: # sdk/component/conf.go # sdk/component/runner.go

# Conflicts: # sdk/component/store/local/store.go # sdk/component/store/local/storer_test.go

andream16 mentioned this pull request Dec 6, 2024

Implement persistent findings storage - Part 3 #494

Merged

andream16 force-pushed the andream16/feature/saas-610/implement-findings-storage-gateway-reader-and-writer/pt-2 branch 6 times, most recently from fc15309 to 116374d Compare December 10, 2024 16:51

andream16 mentioned this pull request Dec 10, 2024

Implement persistent findings storage - Part 1 #492

Merged

andream16 force-pushed the andream16/feature/saas-610/implement-findings-storage-gateway-reader-and-writer/pt-2 branch 4 times, most recently from cf64439 to 20c6c27 Compare December 11, 2024 17:46

andream16 changed the title ~~[WIP] - Implement persistent findings storage - Part 2~~ Implement persistent findings storage - Part 2 Dec 12, 2024

andream16 marked this pull request as ready for review December 12, 2024 10:01

andream16 requested review from northdpole, ptzianos, dlicheva and PTruscott December 12, 2024 10:01

andream16 self-assigned this Dec 12, 2024