Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SWATCH-3152: Configure swatch-producer-azure to skip bad messages #4104

Merged
merged 1 commit into from
Jan 16, 2025

Conversation

wottop
Copy link
Contributor

@wottop wottop commented Jan 13, 2025

Jira issue: SWATCH-3152

Description

Testing

IQE Test MR:
MR 987

Setup

  1. I used an ephemeral instance and the local IQE proxy

Steps

  1. Run the tests in the IQE MR

Verification

  1. Tests pass
  2. Observe the swatch-azure-producer-service pod for a few minutes to confirm that it does not restart. You will also see the bad and good messages in the logs for that pod.

@wottop wottop marked this pull request as draft January 13, 2025 20:54
@wottop wottop force-pushed the wottop/SWATCH-3152 branch from 2eb4226 to bbbf41a Compare January 13, 2025 21:25
@wottop wottop force-pushed the wottop/SWATCH-3152 branch from bbbf41a to 5747d62 Compare January 14, 2025 13:22
@wottop wottop changed the title SWATCH-3142: Bypass bad Kafka messages in swatch-producer-azure SWATCH-3152: Configure swatch-producer-azure to skip bad messages Jan 14, 2025
@wottop
Copy link
Contributor Author

wottop commented Jan 14, 2025

/retest

@wottop wottop marked this pull request as ready for review January 14, 2025 16:17
@liwalker-rh
Copy link

/retest

1 similar comment
@wottop
Copy link
Contributor Author

wottop commented Jan 14, 2025

/retest

@@ -10,7 +10,7 @@ SPLUNK_HEC_BATCH_SIZE=1000
SPLUNK_HEC_BATCH_INTERVAL=10S
SPLUNK_HEC_RETRY_COUNT=3
SPLUNK_HEC_INCLUDE_EX=true
USAGE_AGGREGATE_FAIL_ON_DESER_FAILURE=true
USAGE_AGGREGATE_FAIL_ON_DESER_FAILURE=false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need to disable this?

Copy link
Contributor Author

@wottop wottop Jan 15, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is used in line 112:
mp.messaging.incoming.billable-usage-hourly-aggregate-in.fail-on-deserialization-failure=${USAGE_AGGREGATE_FAIL_ON_DESER_FAILURE}

Without this setting, bad json kills the pod

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Didn't know about this behaviour. We can read more information in https://es.quarkus.io/guides/kafka#handling-deserialization-failures.
Thanks!

@Sgitario Sgitario self-assigned this Jan 15, 2025
@wottop wottop added QE Pull request should be approved by QE before merge Dev Pull requests that need developer review labels Jan 15, 2025
@Sgitario Sgitario added Dev/approved Pull requests that have been approved by all assigned developers and removed Dev Pull requests that need developer review labels Jan 15, 2025
@liwalker-rh
Copy link

/retest

1 similar comment
@liwalker-rh
Copy link

/retest

@liwalker-rh liwalker-rh self-requested a review January 16, 2025 16:42
@liwalker-rh liwalker-rh self-assigned this Jan 16, 2025
@liwalker-rh liwalker-rh added QE/approved Pull requests that have been approved by all assigned QEs and removed QE Pull request should be approved by QE before merge labels Jan 16, 2025
@wottop wottop merged commit 51fc8fa into main Jan 16, 2025
11 of 13 checks passed
@wottop wottop deleted the wottop/SWATCH-3152 branch January 16, 2025 18:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Dev/approved Pull requests that have been approved by all assigned developers QE/approved Pull requests that have been approved by all assigned QEs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants