Unit testing for throttle

[FredrikAugust]

Hi! I'm currently implementing the throttle transform to prevent huge spikes in logs ingested, and have written it like this (input is kubernetes logs):

  throttling:
    type: throttle
    inputs:
      - ...
    key_field: "{{.kubernetes.pod_uid}}"
    threshold: ${LOG_THROTTLE_THRESHOLD:-1000}
    window_secs: 10

In testing I'm setting the LOG_THROTTLE_THRESHOLD to 1 to ensure the throttling kicks in (I've verified that the actual functionality works as expected).

The unit test looks like this:

  - name: log throttling
    inputs:
      - insert_at: throttling
        type: log
        log_fields:
          kubernetes.pod_uid: "1"
          message: "hello 1"
      - insert_at: throttling
        type: log
        log_fields:
          kubernetes.pod_uid: "1"
          message: "hello 2"
      - insert_at: throttling
        type: log
        log_fields:
          kubernetes.pod_uid: "2"
          message: "hello 3"
    outputs:
      - extract_from: throttling
        conditions:
          - type: vrl
            source: |
              assert!(.message != "hello 2")

But it passes when I set the threshold to >1 as well. Is there any way to specify that I want to not observe the hello 2 message in the output?

[pront]

I don't think this is supported at the moment. I did a quick search, this is a relevant feature request: #20278 .

[FredrikAugust]

Thanks, @pront! Killer support 💪🏻