Tasks API

This section describes the task API for communication with IVIS core. IVIS currently supports only Python for programming tasks (that will likely change in the future); therefore examples will be shown only for this case.

Task code

For our code to be useful in the IVIS framework we need a way to communicate with IVIS core. All the communication is set up in a task's code. There will be explanation for the underlying technical details but you won't use them during actual development as you will be using the IVIS python package. This package significantly simplifies all communication with IVIS core.

API details

Primary attributes

Each job once started receives attributes in JSON format on the standard input, file descriptor 0. Example of data:

{
  "params": {
    "sigSet": "example_set",
    "ts_signal": "ts",
    "source_signal": "source"
  },
  "entities": {
    "signalSets": {
      "example_set": {
        "index": "signal_set_1",
        "name": "Example set",
        "namespace": 1
      }
    },
    "signals": {
      "example_set": {
        "source": {
          "field": "s1",
          "name": "Source of values",
          "namespace": 1
        },
        "ts": {
          "field": "s2",
          "name": "Timestamp",
          "namespace": 1
        }
      }
    }
  },
  "state": {
    "created_set": {
      "index": "signal_set_2",
      "type": "_doc",
      "fields": {
        "created_signal": "s3"
      }
    }
  },
  "owned": {
    "signalSets": {}
  },
  "es": {
    "host": "localhost",
    "port": "9200"
  }
}

There are 5 main JSON objects in incoming data:

• params
• entities
• state
• owned
• es

params are parameters from the job being run that were set previously in GUI of the job's settings page. It is always a pair of parameter's identifier and selected value.

entities have 2 JSON objects, signalSets and signals. Each signal set found in the parameters of the job is listed in the signalSets under his CID with 3 properties, index, that is corresponding index in the ElasticSearch, name and namespace. Each signal found in the parameters is listed in the signals object under CID of the signal set it belongs to under its CID with 3 properties, field, that is field in ElasticSearch in the index of its signal set, name and namespace.

state is job's state stored in the ElasticSearch. Content of state depends completely on what is stored there in the job's code. More on that later.

owned contains all the entities created by this job. This process is described in the requests section.

es contains information about ElasticSearch instance. Under host is host's address and under port is port it is listening on.

Job requests

Job can send request to the IVIS core. Requests are accepted on the file descriptor 3 in JSON format and answer is received on the standard input. There are 2 types of requests job can send:

• store_state
• create_signals

store_state will request storing given state. This state is received on each run start in state object mentioned previously. create_signals will request creating new signal set and signals. Example of a store_state request:

{
 "id": 1,
 "type": "store_state",
 "state": {
   "index": "signal_set_2",
   "type": "_doc",
   "fields": {
     "created_signal": "s3"
   }
 }
}

and received answer:

{
  "id": 1
}

Each request has type, either store_state or create_signals, that determines requested action. If there is id present in the request it will be copied to the answer unless the JSON format was incorrect. If there is no error present, request succeeded. Otherwise error with error message will be present in the answer:

{
  "id": 1,
  "error": "Request failed"
}

In the store_state request everything in state object will be stored.

create_signals request looks like this:

{
  "type": "create_signals",
  "signalSets": {
      "cid" : "created_set",
      "name" : "Example set" ,
      "namespace": 1,
      "description" : "Documentation example signal set" ,
      "signals": [
        {
          "cid": "created_signal",
          "name": "Example signal",
          "description": "Documentation example signal set",
          "namespace": 1,
          "type": "text",
          "indexed": false,
          "settings": {} 
        }       
      ]
  }
}

and received answer:

{
  "created_set": {
    "index": "signal_set_2",
    "type": "_doc",
    "fields": {
      "created_signal": "s3"
     }
  }
}

It is possible to request multiple signal sets at once:

{
  "type": "create_signals",
  "signalSets": [
    {
      "cid" : "created_set1",
      "namespace": 1,
      "signals": [
      ]
    },
    {
      "cid" : "created_set2",
      "namespace": 1,
      "signals": [
      ]
    }
  ]
}

answer:

{
  "created_set1": {
    "index": "signal_set_23",
    "type": "_doc",
    "fields": {
     }
  },
  "created_set2": {
    "index": "signal_set_24",
    "type": "_doc",
    "fields": {
     }
  }
}

It is also possible to request creation of signals in an existing signal set:

{
  "type": "create_signals",
  "signals": {
    "existing_signal_set_cid": [
      {
        "cid": "created_signal",
        "name": "Example signal",
        "description": "Documentation example signal set",
        "namespace": 1,
        "type": "text",
        "indexed": false,
        "settings": {} 
      }   
    ]   
  }
}

answer:

{
  "existing_signal_set_cid": {
    "index": "signal_set_16",
    "type": "_doc",
    "fields": {
      "created_signal": "s25"
     }
  }
}

Once the requested signal sets or signals are created they appear in the owned attribute:

"owned": {
  "signalSets": {
    "created_set1": {}
  }
}

IVIS python package

Instead of using the described API directly we will utilize IVIS python package. This library simplifies communication with the IVIS core. First we have to import it:

from ivis import ivis

Previously mentioned attributes are available as properties on ivis object. The es attribute is elasticsearch property and is isntantiated Elasticsearch class from the elasticsearch package.

es = ivis.elasticsearch
state = ivis.state
params = ivis.params
entities = ivis.entities
owned = ivis.owned

There are several helper functions for task's requests. For creating signal sets and signals there are:

ivis.create_signal_set(cid, namespace, name=None, description=None, record_id_template=None, signals=None)
ivis.create_signal(signal_set_cid, cid, namespace, type, name=None, description=None, indexed=None, settings=None, weight_list=None, eight_edit=None, **extra_keys)

These will create signal set and signal with given properties respectively. There is also functions for creating multiple at the same time.

ivis.create_signals(signal_sets=None, signals=None)

At least one of signal_sets or signals have to be set. There is also function for storing state.

ivis.store_state(state)