Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] Fix Flaky Test in test/sample_random/processor-spec.ts #988

Open
sotojn opened this issue Jan 17, 2025 · 1 comment
Open

[bug] Fix Flaky Test in test/sample_random/processor-spec.ts #988

sotojn opened this issue Jan 17, 2025 · 1 comment
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@sotojn
Copy link
Contributor

sotojn commented Jan 17, 2025

We've noticed a test in test/sample_random/processor-spec.ts thats been very flaky lately and happens more often then we'd like. Here is a failed pipeline:

https://github.com/terascope/standard-assets/actions/runs/12832375172/job/35785011148

It should return no data but instead returns 1 somehow. This could be a change to a dependency with different behavior or some sort of race condition.

@sotojn sotojn added bug Something isn't working help wanted Extra attention is needed labels Jan 17, 2025
@busma13
Copy link
Contributor

busma13 commented Jan 17, 2025

It looks like an off by one error in the processor's call to random(). It should return a doc if probability_to_keep is between 1 and 100, not 0 and 99.

godber pushed a commit that referenced this issue Jan 17, 2025
This PR makes the following changes:
- Fixes a bug in the `sample_random` processor when
`probability_to_keep` is set to 0%. If the random number generator
returns 0 then a record would be returned ( 0 <= 0 is true ). Setting
the `random()` function's minimum value to 1 ensures that no records
will ever be returned ( 1 <= 0 is false).
- Update docs
- Add test with large dataset and `probability_to_keep` set to 0%
- bump standard asset from v1.3.1 to v1.3.2

ref: #988
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Development

No branches or pull requests

2 participants