The ngsildclient library

Overview

ngsildclient is a Python library dedicated to NGSI-LD.

It combines :

• a toolbox to create and modify NGSI-LD entities effortlessly
• a NGSI-LD API client to interact with a Context Broker

Features

Build NGSI-LD entities

ngsildclient aims at :

• programmatically generate NGSI-LD entities
• load entities from JSON-LD payloads

Four primitives are provided prop(), gprop(), tprop(), rel() to build respectively a Property, GeoProperty, TemporalProperty and Relationship.

An Entity is backed by a Python dictionary that stores the JSON-LD payload. The library operates the mapping between the Entity's attributes and their JSON-LD counterpart, allowing to easily manipulate NGSI-LD value and metadata directly in Python.

Features list

• primitives to build properties and relationships (chainable)
• benefit from uri naming convention, omit scheme and entity's type, e.g. parking = Entity("OffStreetParking", "Downtown1")
• support dot-notation facility, e.g. reliability = parking["availableSpotNumber.reliability"]
• easily manipulate a property's value, e.g. reliability.value = 0.8
• easily manipulate a property's metadata, e.g. reliability.datasetid = "dataset1"
• support nesting
• support multi-attribute
• load/save to file
• load from HTTP
• load well-known sample entities, e.g. parking = Entity.load(SmartDataModels.SmartCities.Parking.OffStreetParking)
• provide helpers to ease building some structures, e.g. PostalAddress
• pretty-print entity and properties

Interact with the Context Broker

Two clients are provided, Client and AsyncClient respectively for synchronous and asynchronous modes.

Prefer the synchronous one when working in interactive mode, for example to explore and visualize context data in a Jupyter notebook. Prefer the async one if you're looking for performance, for example to develop a real-time NGSI-LD Agent with a high data-acquisition frequency rate.

Features list

• synchronous and asynchronous clients
• support batch operations
• support pagination : transparently handle pagination (sending as many requests as needed under the hood)
• support auto-batch : transparently divide into many batch requests if needed
• support queries and alternate (POST) queries
• support temporal queries
• support pandas dataframe as a temporal query result
• support subscriptions
• find subscription conflicts
• SubscriptionBuilder to help build subscriptions
• auto-detect broker vendor and version
• support follow relationships (chainable), e.g. camera = parking.follow("availableSpotNumber.providedBy")

Getting started

Create our first parking Entity

The following code snippet builds the OffstreetParking sample entity from the ETSI documentation.

from datetime import datetime
from ngsildclient import Entity

PARKING_CONTEXT = "https://raw.githubusercontent.com/smart-data-models/dataModel.Parking/master/context.jsonld"

e = Entity("OffStreetParking", "Downtown1")
e.ctx.append(PARKING_CONTEXT)
e.prop("name", "Downtown One")
e.prop("availableSpotNumber", 121, observedat=datetime(2022, 10, 25, 8)).anchor()
e.prop("reliability", 0.7).rel("providedBy", "Camera:C1").unanchor()
e.prop("totalSpotNumber", 200).loc(41.2, -8.5)

Let's print the JSON-LD payload.

e.pprint()

The result is available here.

Persist our parking in the Context Broker

The following example assumes that an Orion-LD context broker is running on localhost.
A docker-compose config file file is provided for that purpose.

from ngsildclient import Client

client = Client(port=8026, port_temporal=8027)
client.create(e)

Increase our parking occupancy as the day goes on

Each hour ten more parkings spots are occupied, until 8 p.m.

from datetime import timedelta

prop = e["availableSpotNumber"]
for _ in range(12):
    prop.observedat += timedelta(hours=1)
    prop.value -= 10
    client.update(e)

Retrieve our parking

Get back our parking from the broker and display its availableSpotNumber property.

parking = client.get("OffStreetParking:Downtown1", ctx=PARKING_CONTEXT)
parking["availableSpotNumber"].pprint()

Only one available parking spot remains at 8 p.m.

{
    "type": "Property",
    "value": 1,
    "observedAt": "2022-10-25T20:00:00Z",
    "reliability": {
        "type": "Property",
        "value": 0.7
    },
    "providedBy": {
        "type": "Relationship",
        "object": "urn:ngsi-ld:Camera:C1"
    }
}

Request the Temporal Representation of our parking

For convenience we retrieve it as a pandas dataframe.

If you don't have pandas installed, just omit the as_dataframe argument and get JSON instead.

df = client.temporal.get(e, ctx=PARKING_CONTEXT, as_dataframe=True)

Let's display the three last rows.

df.tail(3)

	OffStreetParking	observed	availableSpotNumber
10	Downtown1	2022-10-25 18:00:00+00:00	21
11	Downtown1	2022-10-25 19:00:00+00:00	11
12	Downtown1	2022-10-25 20:00:00+00:00	1

Let's throw in a more realistic parking management system

Let us move from our first example to the more realistic parking example provided by the Smart Data Models Program.

from ngsildclient import SmartDataModels

parking = Entity.load(SmartDataModels.SmartCities.Parking.OffStreetParking)

Once loaded we can manipulate our new parking the same way we've done until now.
Let's see how it is occupied.

n_total = parking["totalSpotNumber"].value
n_occupied = parking["occupiedSpotNumber"].value
n_avail= parking["availableSpotNumber"].value
print(n_total, n_occupied, n_avail)

This parking has 414 parking slots. 282 are occupied. 132 are available.
In order to complete our parking system we would like to add 414 spots to our datamodel.
Let's create a reference parking spot to be used as a template.

spot = Entity("ParkingSpot", "OffStreetParking:porto-ParkingLot-23889:000")
spot.prop("status", "free")
spot.rel("refParkingSite", parking)

Let's clone this spot 414 times, assign a disctinct id to each one and occupy the 282 first spots.
This is a simplistic strategy but enough to keep the parking system consistent.

spots = spot * n_total
for i, spot in enumerate(spots):
    spot.id = f"{spot.id[:-3]}{i+1:03}"
    if i < n_occupied:
        spot["status"].value = "occupied"

We now establish the relationship between the parking and its spots by adding a new attribute to the parking.
Having a mutual relationship is not necessarily needed. It depends on how we want to navigate in our datamodel.
Let's do it for the sake of example.

from ngsildclient import MultAttrValue

mrel = MultAttrValue()
for spot in spots:
    mrel.add(spot, datasetid=f"Dataset:{spot.id[-26:]}")
parking.rel("refParkingSpot", mrel)

To sum up we have obtained 415 entities : 1 parking and 414 spots.
Make a single list of these parts and save it into a file.

datamodel = sum(([parking], spots), [])  # flatten lists
Entity.save_batch(datamodel, "parking_system.jsonld")

The result is available here.
Time now to populate our parking system in the broker.

client.upsert(datamodel)

Check everything is fine by asking the broker for the number of occupied spots.
Eventually close the client.

client.count("ParkingSpot", q='refParkingSite=="urn:ngsi-ld:OffStreetParking:porto-ParkingLot-23889";status=="occupied"')  # 282
client.close()

Let's go further

1. Develop a NGSI-LD Agent

   • Collect incoming data from parking IoT (ground sensors, cameras) and the parking system API
   • Clean data, process data and convert to NGSI-LD entities
   • Create and update entities into the NGSI-LD broker in real-time

2. Subscribe to events

   • Create a subscription to be informed when parking occupation exceeds 90%
   • The software that listens to these highly-occupied parking entities can also be a NGSI-LD Agent

   
   Example : programmatically subscribe to events

   from ngsildclient import SubscriptionBuilder
   
   subscr = SubscriptionBuilder("https://parkingsystem.example.com:8000/subscription/high-occupancy")
       .description("Notify me of high occupancy on parking porto-23889")
       .select_type("OffStreetParking")
       .watch(["occupancy"])
       .query('occupancy>0.9;controlledAsset=="urn:ngsi-ld:OffStreetParking:porto-ParkingLot-23889"')
       .build()
   client.subscriptions.create(subscr)

Where to get it

The source code is currently hosted on GitHub at : https://github.com/Orange-OpenSource/python-ngsild-client

Binary installer for the latest released version is available at the Python package index.

Installation

ngsildclient requires Python 3.9+.

pip install ngsildclient

Documentation

User guide is available on Read the Docs.

Refer to the Cookbook chapter that provides many HOWTOs to :

• develop various NGSI-LD Agents collecting data from heterogeneous datasources
• forge NGSI-LD sample entities from the Smart Data Models initiative

License

Apache 2.0