Send state_id over time to have availability

[Azef1]

This Pull request send availability for hosts and services in Graphite over time.
It's useful to see when an host/service is down over time.

How it's works ?
The broker receive all data from all check. A function for host read data about state and other function do same for service.
For HOSTS i added:

 def manage_host_check_result_brok(self, b):
    state_id = b.data['state_id'] #Read state_id, it's a number
[...]
#Send state to Graphite        
        state_query = []
        state_query.append("%s.available %s %d" % (path, state_id, check_time))
        state_packet = '\n'.join(state_query) + '\n'
        #Query example: myshost.available 1 ​1458774003
        #logger.error("------ %s", state_packet)
        try:
                self.send_packet(state_packet) 
        except IOError:
                logger.error("[Graphite broker] Failed sendind state o the Graphite Carbon.")
[...]

Same logic for services

The result with Grafana and Graphite as datasource

[mohierf]

You should make this feature configurable in the module configuration file. Sending state for all hosts and services may be overloading an installed Graphite instance ...

[Azef1]

Thx for your suggestion. I added some options to enable or disable this feature and you will be able to choose sending state for all hosts OR\AND all services.

I am agree with you "Sending state for all hosts or\and all services may be overloading" for Graphite.
Currently, i send state every time a check is executed. This method is too aggressive if check_interval is aggressive (eg: if check_interval 10, all 10 seconds a query is sending to Graphite)
In next release, i will add a function to send state to Graphite only if previous state is different from current state. So, i send a packet to Graphite only if state change.

Other suggestions are welcome.

[mohierf]

+1 for the options. I think that this feature should be disabled as default.

Nice idea to send the state only if it changed. As of my memory, you can easily know if the state chaned thanks to the received brok data ... you should find a state and a last_state information in the brok data :)

[Azef1]

So, I disabled this feature as default.
And i send state to Graphite only if last_state_id != state_id
You can test it

[Azef1]

With graphite, I replicated same result than Grafana:

Here Service is down between ~9h00 to 15h.

Do you think this feature could be "official" in master branch or other branch ?
Have you test this feature ?
It will be nice to modify ui-graphite module to embed this graph in WebUI (with other size height: 100px width: 100%).