Add config for V18 + allow router + util to generate config for new metrics

[rafitadiaz]

• Add a configuration template for the V0.18 of druid.
• Add router as allowed deamon
• Add a utility to generate configuration block from the log of failed metrics.
  So basically it converts this:

 DEBUG:druid_exporter.collector:The following datapoint is not supported, either because the 'feed' field is not 'metrics' or the metric itself is not supported: {'feed': 'metrics', 'timestamp': '2020-05-07T09:57:08.983Z', 'service': 'druid/router', 'host': '10.132.0.27:8888', 'version': '0.18.0', 'metric': 'query/time', 'value': 7, 'context': {'queryId': '2c08a196-9c11-49c3-a5e7-b41357388f24', 'timeout': 40000}, 'dataSource': 'datasource_test_1', 'duration': 'PT9223372036854775.807S', 'hasFilters': 'false', 'id': '2c08a196-9c11-49c3-a5e7-b41357388f24', 'interval': ['-146136543-09-08T08:23:32.096Z/146140482-04-24T15:36:27.903Z'], 'remoteAddress': '10.0.3.217', 'success': 'true', 'type': 'timeBoundary'}

into this:

----------------
    Found metric: 
    {
        "context": {
            "queryId": "2c08a196-9c11-49c3-a5e7-b41357388f24", 
            "timeout": 40000
        }, 
        "dataSource": "datasource_test_1", 
        "duration": "PT9223372036854775.807S", 
        "feed": "metrics", 
        "hasFilters": "false", 
        "host": "10.132.0.27:8888", 
        "id": "2c08a196-9c11-49c3-a5e7-b41357388f24", 
        "interval": [
            "-146136543-09-08T08:23:32.096Z/146140482-04-24T15:36:27.903Z"
        ], 
        "metric": "query/time", 
        "remoteAddress": "10.0.3.217", 
        "service": "druid/router", 
        "success": "true", 
        "timestamp": "2020-05-07T09:57:08.983Z", 
        "type": "timeBoundary", 
        "value": 7, 
        "version": "0.18.0"
    }

    This is a metric for: druid/router
    Adding dataSource as label for prometheus.
    {
        "query/time": {
            "buckets": [
                "10", 
                "100", 
                "500", 
                "1000", 
                "2000", 
                "3000", 
                "5000", 
                "7000", 
                "10000", 
                "inf", 
                "sum"
            ], 
            "description": "Milliseconds taken to complete a query.", 
            "labels": [
                "dataSource"
            ], 
            "prometheus_metric_name": "druid_query_time", 
            "type": "histogram"
        }
    }

So basically you just need to review if the config is OK and copy it in the right place inside the Json file.

[elukey]

@rafitadiaz thanks a lot for the pull request, it looks awesome! Just wanted to say that I am aware of it but didn't have the time to review it yet, I promise some feedback next week!

[elukey]

I have reviewed the patch and I have few ideas, let me know what you think about them.

I would split the pull request into two/three parts:

• add router as allowed daemon
• add V18 config
• add utils section

About the first two: this is a mirror only repo sadly, so I cannot merge anything from GH. The real repo is on Gerrit Wikimedia (gerrit.wikimedia.org), you can register your user, clone the repo and execute something like git push origin HEAD:refs/for/master to create a pull request that I can merge. Or I can create one and give you credits on the commit msg, as you prefer :)

About the utils section: I am wondering if, in the interest of keeping this repo as slim as possible, we could create a new repo for the tool (maybe under you user in GH) and add a "Tools" section in this repo's readme, linking to your tool (with some description etc..). What do you think? If more people add tools (that would be great) it will be more scalable, since different code bases could have different dev workflows.

[rafitadiaz]

Sure, I would be happy to do it.
Next week I'll take some time to split those PR. If gerrit is the place let's do it there.
About split the utils in a separated repo. I'm not sure if this have enough weight to be a repo by its own. Is just a very simple script to help to understand the missing metrics.
But if you prefer not to add it I can create a independent repo for this.

[elukey]

@rafitadiaz Thanks a lot! For the moment I'd prefer a separate repo if possible, will wait for the other pull requests then :)