No Tests Discovered When Specifying Negative Metrics Values

[Evesy]

Hi,

I have the below tests in place which don't seem to handle the existence of negative numbers in the metric values:

When I replace -10 & -20 with 0 values the tests runs OK:

=== RUN   test_at_5m
--- PASS: test_at_5m (0.00s)
=== RUN   test_at_15m
--- PASS: test_at_15m (0.00s)
PASS

However with the negative values in place:

WARNING: No tests discovered. Exiting...

Test file:

name: test
rules:
  fromFile: ../files/alerting.rules.yaml
fixtures:
  - duration: 5m
    metrics:
      - ingress_controller_ssl_expire_time_seconds{host="expiringsoon.domain",instance="10.202.8.83:10254",job="kubernetes-pods",namespace="ingress-nginx",pod_name="ingress-nginx-internal-controller-fb7c6bbbb-9nvbj",pod_template_hash="963726666",purpose="ingress-nginx"} 5000 2500 1000
      - ingress_controller_ssl_expire_time_seconds{host="expiringsoon.domain",instance="10.202.8.84:10254",job="kubernetes-pods",namespace="ingress-nginx",pod_name="ingress-nginx-internal-controller-fb7c6bbbb-9nvbk",pod_template_hash="963726666",purpose="ingress-nginx"} 5000 2500 1000
      - ingress_controller_ssl_expire_time_seconds{host="expiringsoon.domain",instance="10.202.8.85:10254",job="kubernetes-pods",namespace="ingress-nginx",pod_name="ingress-nginx-internal-controller-fb7c6bbbb-9nvbl",pod_template_hash="963726666",purpose="ingress-nginx"} 5000 2500 1000
      - ingress_controller_ssl_expire_time_seconds{host="expired.domain",instance="10.202.8.83:10254",job="kubernetes-pods",namespace="ingress-nginx",pod_name="ingress-nginx-internal-controller-fb7c6bbbb-9nvbj",pod_template_hash="963726666",purpose="ingress-nginx"} 0 -10 -20
      - ingress_controller_ssl_expire_time_seconds{host="expired.domain",instance="10.202.8.84:10254",job="kubernetes-pods",namespace="ingress-nginx",pod_name="ingress-nginx-internal-controller-fb7c6bbbb-9nvbk",pod_template_hash="963726666",purpose="ingress-nginx"} 0 -10 -20
      - ingress_controller_ssl_expire_time_seconds{host="expired.domain",instance="10.202.8.85:10254",job="kubernetes-pods",namespace="ingress-nginx",pod_name="ingress-nginx-internal-controller-fb7c6bbbb-9nvbl",pod_template_hash="963726666",purpose="ingress-nginx"} 0 -10 -20
assertions:
  - at: 5m
    expected:
      - alertname: SSLCertExpiry
        alertstate: pending
        severity: warning
        host: expiringsoon.domain
      - alertname: SSLCertExpiry
        alertstate: pending
        severity: warning
        host: expired.domain
      - alertname: SSLCertExpired
        alertstate: pending
        severity: critical
        host: expired.domain
  - at: 15m
    expected:
      - alertname: SSLCertExpiry
        alertstate: firing
        severity: warning
        host: expiringsoon.domain
      - alertname: SSLCertExpiry
        alertstate: firing
        severity: warning
        host: expired.domain
      - alertname: SSLCertExpired
        alertstate: firing
        severity: critical
        host: expired.domain

[kevinjqiu]

Ahh I missed an error reporting in the main code. After I added that in, this is the error I got:

2018/07/05 14:13:49 parse error at line 6, char 265: unexpected <op:-> in series values, expected "x"

Basically, the prom test case parser did not expect a negative number to be used as metrics value. I will confirm this with the upstream prometheus code.

EDIT: added the code to report test case generation errors and it's now on master.

[kevinjqiu]

@Evesy I've reported a bug upstream prometheus/prometheus#4356