chore: updating pubsub system #3646

JaydipGabani · 2024-10-15T05:50:11Z

What this PR does / why we need it:

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):
Fixes #1037 #3497

Special notes for your reviewer:

codecov-commenter · 2024-10-15T22:06:18Z

Codecov Report

Attention: Patch coverage is 52.83019% with 75 lines in your changes missing coverage. Please review.

Project coverage is 47.71%. Comparing base (3350319) to head (1f2c01b).
Report is 250 commits behind head on master.

Files with missing lines	Patch %	Lines
pkg/export/dapr/fake_dapr_client.go	24.13%	22 Missing ⚠️
pkg/export/dapr/dapr.go	65.95%	14 Missing and 2 partials ⚠️
pkg/export/testdriver/testdriver.go	36.84%	11 Missing and 1 partial ⚠️
pkg/controller/export/export_config_controller.go	8.33%	11 Missing ⚠️
pkg/export/system.go	79.54%	6 Missing and 3 partials ⚠️
pkg/audit/manager.go	0.00%	5 Missing ⚠️

❗ There is a different number of reports uploaded between BASE (3350319) and HEAD (1f2c01b). Click for more details.

HEAD has 1 upload less than BASE

Flag BASE (3350319) HEAD (1f2c01b)

unittests 2 1

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3646      +/-   ##
==========================================
- Coverage   54.49%   47.71%   -6.79%     
==========================================
  Files         134      234     +100     
  Lines       12329    19864    +7535     
==========================================
+ Hits         6719     9478    +2759     
- Misses       5116     9496    +4380     
- Partials      494      890     +396

Flag	Coverage Δ
unittests	`47.71% <52.83%> (-6.79%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

JaydipGabani · 2024-10-23T01:32:19Z

Still working on doc changes.

JaydipGabani · 2024-12-05T23:31:13Z

pkg/controller/export/export_config_controller.go

@@ -26,37 +26,37 @@ import (
 )

 var (
-	PubsubEnabled = flag.Bool("enable-pub-sub", false, "(alpha) Enabled pubsub to publish messages")
+	ExportEnabled = flag.Bool("enable-pub-sub", false, "(alpha) Enabled pubsub to publish messages")


Do we want to rename the flag to remove pub-sub word?

May as well while it's still alpha

should we provide a warning for the user if we change the name? we can remove it after a release?

What do you mean? How would we provide warning for changing a name before release?

JaydipGabani · 2024-12-31T19:26:10Z

Fixes #3497

maxsmythe · 2025-01-16T02:44:54Z

pkg/controller/export/export_config_controller.go

@@ -114,22 +114,22 @@ func (r *Reconciler) Reconcile(ctx context.Context, request reconcile.Request) (
 	}

 	if len(cfg.Data) == 0 {
-		return reconcile.Result{}, fmt.Errorf(fmt.Sprintf("data missing in configmap %s, unable to configure respective pubsub", request.NamespacedName))
+		return reconcile.Result{}, fmt.Errorf(fmt.Sprintf("data missing in configmap %s, unable to establish connection", request.NamespacedName))


is it a "connection" necessarily?

"link" may be? I just couldn't come up with better alternative at the time.

"unable to configure exporter"?

maxsmythe · 2025-01-16T02:46:34Z

pkg/controller/export/export_config_controller.go

 	}
-	if _, ok := cfg.Data["provider"]; !ok {
-		return reconcile.Result{}, fmt.Errorf(fmt.Sprintf("missing provider field in configmap %s, unable to configure respective pubsub", request.NamespacedName))
+	if _, ok := cfg.Data["driver"]; !ok {


Is changing from config map to CRD coming later?

I can raise a follow up PR, we have the design finalized as per last discussion.

sozercan

minor comments, otherwise LGTM

sozercan · 2025-01-23T19:29:26Z

pkg/controller/export/export_config_controller.go

 }

 func (a *Adder) Add(mgr manager.Manager) error {
-	if !*PubsubEnabled {
+	if !*ExportEnabled {
 		return nil
 	}
 	log.Info("Warning: Alpha flag enable-pub-sub is set to true. This flag may change in the future.")


update this too if we change the flag name

sozercan · 2025-01-23T19:29:41Z

pkg/controller/export/export_config_controller.go

@@ -114,22 +114,22 @@ func (r *Reconciler) Reconcile(ctx context.Context, request reconcile.Request) (
 	}

 	if len(cfg.Data) == 0 {
-		return reconcile.Result{}, fmt.Errorf(fmt.Sprintf("data missing in configmap %s, unable to configure respective pubsub", request.NamespacedName))
+		return reconcile.Result{}, fmt.Errorf(fmt.Sprintf("data missing in configmap %s, unable to establish connection", request.NamespacedName))


"unable to configure exporter"?

sozercan · 2025-01-23T19:31:59Z

website/docs/export-driver-walkthrough.md

+title: Export Interface/Driver walkthrough
+---
+
+This guide provides an overview of the driver interface, including details of its structure and functionality. Additionally, it offers instructions on adding a new driver and utilizing different backends to export violations.


Suggested change

This guide provides an overview of the driver interface, including details of its structure and functionality. Additionally, it offers instructions on adding a new driver and utilizing different backends to export violations.

This guide provides an overview of the driver interface, including details of its structure and functionality. Additionally, it offers instructions on adding a new driver and utilizing different backends to export audit violations.

pkg/audit/manager.go

Signed-off-by: Jaydip Gabani <[email protected]>

ritazh · 2025-01-30T06:02:37Z

.github/workflows/dapr-pubsub.yaml

@@ -1,18 +1,18 @@
-name: dapr-pubsub


since you are renaming everything, does this yaml need to be renamed too?

good call, ty!

Updated the file name.

Signed-off-by: Jaydip Gabani <[email protected]>

JaydipGabani changed the title ~~[WIP] chore: updating violation system~~ [WIP] chore: updating pubsub system Oct 21, 2024

JaydipGabani force-pushed the exporter-interface branch from 570505d to 7c48595 Compare October 22, 2024 23:55

JaydipGabani changed the title ~~[WIP] chore: updating pubsub system~~ chore: updating pubsub system Oct 23, 2024

JaydipGabani marked this pull request as ready for review October 23, 2024 01:31

JaydipGabani requested a review from a team as a code owner October 23, 2024 01:31

JaydipGabani commented Dec 6, 2024

View reviewed changes

ritazh added this to the v3.19.0 milestone Dec 11, 2024

JaydipGabani mentioned this pull request Dec 31, 2024

Modify Pub/Sub interface to common exporter interface #3497

Open

maxsmythe reviewed Jan 16, 2025

View reviewed changes

JaydipGabani force-pushed the exporter-interface branch from bf0c5cf to 1f2905e Compare January 24, 2025 22:12

JaydipGabani requested review from sozercan, maxsmythe and ritazh January 24, 2025 22:13

sozercan reviewed Jan 28, 2025

View reviewed changes

updating pubsub system

64e2ae5

Signed-off-by: Jaydip Gabani <[email protected]>

JaydipGabani force-pushed the exporter-interface branch from 1b578cb to 2fba5c2 Compare January 28, 2025 19:39

JaydipGabani requested a review from sozercan January 29, 2025 23:40

replacing pub-sub with export everywhere

95b5ab7

Signed-off-by: Jaydip Gabani <[email protected]>

JaydipGabani force-pushed the exporter-interface branch from cabc202 to 95b5ab7 Compare January 30, 2025 00:17

fixing helm variable name enableViolationExport

74d9a44

Signed-off-by: Jaydip Gabani <[email protected]>

ritazh reviewed Jan 30, 2025

View reviewed changes

JaydipGabani added 2 commits January 30, 2025 19:10

renaming dapr-pubsub -> dapr-export

b48385a

Signed-off-by: Jaydip Gabani <[email protected]>

Merge branch 'master' into exporter-interface

f4d5631

JaydipGabani requested a review from ritazh January 30, 2025 22:47

JaydipGabani force-pushed the exporter-interface branch from 68424cf to f4d5631 Compare January 31, 2025 01:56

JaydipGabani added 2 commits February 3, 2025 10:41

Merge branch 'master' into exporter-interface

8749ea6

Merge branch 'master' into exporter-interface

3d124af

Merge remote-tracking branch 'gatekeeper/master' into exporter-interface

1f2c01b

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: updating pubsub system #3646

chore: updating pubsub system #3646

JaydipGabani commented Oct 15, 2024 •

edited

Loading

codecov-commenter commented Oct 15, 2024 •

edited

Loading

JaydipGabani commented Oct 23, 2024

JaydipGabani Dec 5, 2024

maxsmythe Jan 16, 2025

sozercan Jan 23, 2025 •

edited

Loading

JaydipGabani Jan 24, 2025

JaydipGabani commented Dec 31, 2024

maxsmythe Jan 16, 2025

JaydipGabani Jan 16, 2025

sozercan Jan 23, 2025

maxsmythe Jan 16, 2025

JaydipGabani Jan 16, 2025

sozercan left a comment

sozercan Jan 23, 2025

sozercan Jan 23, 2025

sozercan Jan 23, 2025

ritazh Jan 30, 2025

JaydipGabani Jan 30, 2025

JaydipGabani Jan 30, 2025

	This guide provides an overview of the driver interface, including details of its structure and functionality. Additionally, it offers instructions on adding a new driver and utilizing different backends to export violations.
	This guide provides an overview of the driver interface, including details of its structure and functionality. Additionally, it offers instructions on adding a new driver and utilizing different backends to export audit violations.

chore: updating pubsub system #3646

Are you sure you want to change the base?

chore: updating pubsub system #3646

Conversation

JaydipGabani commented Oct 15, 2024 • edited Loading

codecov-commenter commented Oct 15, 2024 • edited Loading

Codecov Report

JaydipGabani commented Oct 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sozercan Jan 23, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JaydipGabani commented Dec 31, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sozercan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JaydipGabani commented Oct 15, 2024 •

edited

Loading

codecov-commenter commented Oct 15, 2024 •

edited

Loading

sozercan Jan 23, 2025 •

edited

Loading