Architecture

This is an introductory tutorial to the FIWARE Platform. We will start with the data from a supermarket chain store finder and create a very simple “Powered by FIWARE” application by passing in the address and location of each store as context data to the FIWARE context broker.

The tutorial uses cUrl commands throughout, but is also available as Postman documentation

🇯🇵 このチュートリアルは日本語でもご覧いただけます。
🇵🇹 Este tutorial também está disponível em português
🇪🇸 Este tutorial también está disponible en español

Architecture

Our demo application will only make use of one FIWARE component - the Orion Context Broker. Usage of the Orion Context Broker is sufficient for an application to qualify as “Powered by FIWARE”.

Currently, the Orion Context Broker relies on open source MongoDB technology to keep persistence of the context data it holds. Therefore, the architecture will consist of two elements:

The Orion Context Broker which will receive requests using NGSI-v2
The underlying MongoDB database:
- Used by the Orion Context Broker to hold context data information such as data entities, subscriptions and registrations

Since all interactions between the two elements are initiated by HTTP requests, the entities can be containerized and run from exposed ports.

Prerequisites

Operating System

This tutorial is written for Linux environments, so the commands would need to be heavily adapted if using Windows.

Docker

To keep things simple, both components will be run using Docker. Docker is a container technology which allows different components to be deployed into their respective isolated environments.

To install Docker on Windows, follow the instructions here
To install Docker on Mac, follow the instructions here
To install Docker on Linux, follow the instructions here

Docker Compose (Optional)

Docker Compose is a tool for defining and running multi-container Docker applications. A YAML file is used to configure the required services for the application. This means all container services can be brought up in a single command. Docker Compose is installed by default as part of Docker for Windows and Docker for Mac, however Linux users will need to follow the instructions found here.

You can check your current Docker and Docker Compose versions using the following commands:

docker-compose -v
docker version

Please ensure that you are using Docker version 20.10 or higher and Docker Compose 1.29 or higher and upgrade if necessary.

Starting the containers

Option 1) Using Docker commands directly

First pull the necessary Docker images from Docker Hub and create a network for our containers to connect to:

docker pull mongo:4.4
docker pull fiware/orion
docker network create fiware_default

A Docker container running a MongoDB database can be started and connected to the network with the following command:

docker run -d --name=mongo-db --network=fiware_default \
  --expose=27017 mongo:4.4 --bind_ip_all

The Orion Context Broker can be started and connected to the network with the following command:

docker run -d --name fiware-orion -h orion --network=fiware_default \
  -p 1026:1026  fiware/orion -dbhost mongo-db

Note: If you want to clean up and start again you can do so with the following commands
docker stop fiware-orion
docker rm fiware-orion
docker stop mongo-db
docker rm mongo-db
docker network rm fiware_default

Option 2) Using Docker Compose

All services can be initialised from the command-line using the docker-compose command or the newer docker compose command (without the hyphen -) found in Compose V2. Please clone the repository and create the necessary images by running the commands as shown:

git clone https://github.com/FIWARE/tutorials.Getting-Started.git
cd tutorials.Getting-Started
git checkout NGSI-v2

export $(cat .env | grep "#" -v)
docker compose -p fiware up -d

Note: If you want to clean up and start again you can do so with the following command:
docker compose -p fiware down

Creating your first "Powered by FIWARE" app

Checking the service health

You can check if the Orion Context Broker is running by making an HTTP request to the exposed port:

1️⃣ Request:

curl -X GET \
  'http://localhost:1026/version'

Response:

The response will look similar to the following:

{
    "orion": {
        "version": "3.0.0",
        "uptime": "0 d, 0 h, 17 m, 19 s",
        "git_hash": "d6f8f4c6c766a9093527027f0a4b3f906e7f04c4",
        "compile_time": "Mon Apr 12 14:48:44 UTC 2021",
        "compiled_by": "root",
        "compiled_in": "f307ca0746f5",
        "release_date": "Mon Apr 12 14:48:44 UTC 2021",
        "machine": "x86_64",
        "doc": "https://fiware-orion.rtfd.io/en/3.0.0/",
        "libversions": {
            "boost": "1_66",
            "libcurl": "libcurl/7.61.1 OpenSSL/1.1.1g zlib/1.2.11 nghttp2/1.33.0",
            "libmicrohttpd": "0.9.70",
            "openssl": "1.1",
            "rapidjson": "1.1.0",
            "mongoc": "1.17.4",
            "bson": "1.17.4"
        }
    }
}

What if I get a Failed to connect to localhost port 1026: Connection refused Response?

If you get a Connection refused response, the Orion Content Broker cannot be found where expected for this tutorial - you will need to substitute the URL and port in each cUrl command with the corrected IP address. All the cUrl commands in this tutorial assume that orion is available on localhost:1026.

Try the following remedies:

To check that the docker containers are running try the following:
docker ps
You should see two containers running. If orion is not running, you can restart the containers as necessary. This command will also display open port information.

If you have installed docker-machine and Virtual Box, the orion docker container may be running from another IP address - you will need to retrieve the virtual host IP as shown:
curl -X GET \
 'http://$(docker-machine ip default):1026/version'
Alternatively run all your cUrl commands from within the container network:
docker run --network fiware_default --rm appropriate/curl -s \
 -X GET 'http://orion:1026/version'

Creating Context Data

At its heart, FIWARE is a system for managing context information, so lets add some context data into the system by creating two new entities (stores in Berlin). Any entity must have the attributes id and type - additional attributes are optional and will depend on the system being described. Each additional attribute should also have defined type and value attributes.

2️⃣ Request:

curl -iX POST \
  'http://localhost:1026/v2/entities' \
  -H 'Content-Type: application/json' \
  -d '
{
    "id": "urn:ngsi-ld:Store:001",
    "type": "Store",
    "address": {
        "type": "PostalAddress",
        "value": {
            "streetAddress": "Bornholmer Straße 65",
            "addressRegion": "Berlin",
            "addressLocality": "Prenzlauer Berg",
            "postalCode": "10439"
        },
        "metadata": {
            "verified": {
                "value": true,
                "type": "Boolean"
            }
        }
    },
    "location": {
        "type": "geo:json",
        "value": {
             "type": "Point",
             "coordinates": [13.3986, 52.5547]
        }
    },
    "name": {
        "type": "Text",
        "value": "Bösebrücke Einkauf"
    }
}'

3️⃣ Request:

Each subsequent entity must have a unique id for the given type

curl -iX POST \
  'http://localhost:1026/v2/entities' \
  -H 'Content-Type: application/json' \
  -d '
{
    "type": "Store",
    "id": "urn:ngsi-ld:Store:002",
    "address": {
        "type": "PostalAddress",
        "value": {
            "streetAddress": "Friedrichstraße 44",
            "addressRegion": "Berlin",
            "addressLocality": "Kreuzberg",
            "postalCode": "10969"
        },
        "metadata": {
            "verified": {
                "value": true,
                "type": "Boolean"
            }
        }
    },
    "location": {
        "type": "geo:json",
        "value": {
             "type": "Point",
             "coordinates": [13.3903, 52.5075]
        }
    },
    "name": {
        "type": "Text",
        "value": "Checkpoint Markt"
    }
}'

Data Model Guidelines

Although each data entity within your context will vary according to your use case, the common structure within each data entity should be standardized to promote reuse. The full Smart Data model guidelines can be found here. This tutorial demonstrates the usage of the following recommendations:

All terms are defined in American English

Although the value fields of the context data may be in any language, all attributes and types are written using the English language.

Entity type names must start with a Capital letter

In this case we only have one entity type - Store.

Entity IDs should be a URN following NGSI-LD guidelines

NGSI-LD has recently been published as a full ETSI specification. The proposal is that each id is a URN and follows a standard format: urn:ngsi-ld:<entity-type>:<entity-id>. This will mean that every id in the system will be unique.

Data type names should reuse schema.org data types where possible

Schema.org is an initiative to create common structured data schemas. In order to promote reuse, we have deliberately used the Text and PostalAddress type names within our Store entity. Other existing standards such as Open311 (for civic issue tracking) or Datex II (for transport systems) can also be used, but the point is to check for the existence of the same attribute on existing data models and reuse it.

Use camel case syntax for attribute names

The streetAddress, addressRegion, addressLocality and postalCode are all examples of attributes using camel casing.

Location information should be defined using `address` and `location` attributes

We have used an address attribute for civic locations as per schema.org.
We have used a location attribute for geographical coordinates.

Use GeoJSON for codifying geospatial properties

GeoJSON is an open standard format designed for representing simple geographical features. The location attribute has been encoded as a geoJSON Point location.

Attribute Metadata

Metadata is "data about data", it is additionl data used to describe properties of the attribute value itself like accuracy, provider, or a timestamp. Several built-in metadata attribute already exist and these names are reserved

dateCreated (type: DateTime): attribute creation date as an ISO 8601 string.
dateModified (type: DateTime): attribute modification date as an ISO 8601 string.
previousValue (type: any): only in notifications. The value of this metadata is the previous value (to the request triggering the notification) of the associated attribute. The type of this metadata must be the previous type of the associated attribute. If the type/value of previousValue is the same type/value as in the associated attribute, then the attribute has not actually changed its value.
actionType (type: Text): only in notifications. It is included if the attribute to which it is attached was included in the request that triggered the notification. Its value depends on the request operation type: update for updates, append for creation and delete for deletion. Its type is always Text.

One element of metadata can be found within the address attribute: a verified flag indicates whether the address has been confirmed.

Querying Context Data

A consuming application can now request context data by making HTTP requests to the Orion Context Broker. The existing NGSI interface enables us to make complex queries and filter results.

At the moment, for the store finder demo all the context data is being added directly via HTTP requests, however in a more complex smart solution, the Orion Context Broker will also retrieve context directly from attached sensors associated to each entity.

Here are a few examples, in each case the options=keyValues query parameter has been used shorten the responses by stripping out the type elements from each attribute

Obtain entity data by ID

This example returns the data of urn:ngsi-ld:Store:001

4️⃣ Request:

curl -G -X GET \
   'http://localhost:1026/v2/entities/urn:ngsi-ld:Store:001' \
   -d 'options=keyValues'

Response:

Because of the use of the options=keyValues, the response consists of JSON only without the attribute type and metadata elements.

{
    "id": "urn:ngsi-ld:Store:001",
    "type": "Store",
    "address": {
        "streetAddress": "Bornholmer Straße 65",
        "addressRegion": "Berlin",
        "addressLocality": "Prenzlauer Berg",
        "postalCode": "10439"
    },
    "location": {
        "type": "Point",
        "coordinates": [13.3986, 52.5547]
    },
    "name": "Bösebrücke Einkauf"
}

Obtain entity data by type

This example returns the data of all Store entities within the context data The type parameter limits the response to store entities only.

5️⃣ Request:

curl -G -X GET \
    'http://localhost:1026/v2/entities' \
    -d 'type=Store' \
    -d 'options=keyValues'

Response:

Because of the use of the options=keyValues, the response consists of JSON only without the attribute type and metadata elements.

[
    {
        "id": "urn:ngsi-ld:Store:001",
        "type": "Store",
        "address": {
            "streetAddress": "Bornholmer Straße 65",
            "addressRegion": "Berlin",
            "addressLocality": "Prenzlauer Berg",
            "postalCode": "10439"
        },
        "location": {
            "type": "Point",
            "coordinates": [13.3986, 52.5547]
        },
        "name": "Bose Brucke Einkauf"
    },
    {
        "id": "urn:ngsi-ld:Store:002",
        "type": "Store",
        "address": {
            "streetAddress": "Friedrichstraße 44",
            "addressRegion": "Berlin",
            "addressLocality": "Kreuzberg",
            "postalCode": "10969"
        },
        "location": {
            "type": "Point",
            "coordinates": [13.3903, 52.5075]
        },
        "name": "Checkpoint Markt"
    }
]

Filter context data by comparing the values of an attribute

This example returns all stores with the name attribute Checkpoint Markt. Filtering can be done using the q parameter - if a string has spaces in it, it can be URL encoded and held within single quote characters ' = %27

6️⃣ Request:

curl -G -X GET \
    'http://localhost:1026/v2/entities' \
    -d 'type=Store' \
    -d 'q=name==%27Checkpoint%20Markt%27' \
    -d 'options=keyValues'

Response:

Because of the use of the options=keyValues, the response consists of JSON only without the attribute type and metadata elements.

[
    {
        "id": "urn:ngsi-ld:Store:002",
        "type": "Store",
        "address": {
            "streetAddress": "Friedrichstraße 44",
            "addressRegion": "Berlin",
            "addressLocality": "Kreuzberg",
            "postalCode": "10969"
        },
        "location": {
            "type": "Point",
            "coordinates": [13.3903, 52.5075]
        },
        "name": "Checkpoint Markt"
    }
]

Filter context data by comparing the values of a sub-attribute

This example returns all stores found in the Kreuzberg District.

Filtering can be done using the q parameter - sub-attributes are annotated using the dot syntax e.g. address.addressLocality

7️⃣ Request:

curl -G -X GET \
    'http://localhost:1026/v2/entities' \
    -d 'type=Store' \
    -d 'q=address.addressLocality==Kreuzberg' \
    -d 'options=keyValues'

Response:

Because of the use of the options=keyValues, the response consists of JSON only without the attribute type and metadata elements.

[
    {
        "id": "urn:ngsi-ld:Store:002",
        "type": "Store",
        "address": {
            "streetAddress": "Friedrichstraße 44",
            "addressRegion": "Berlin",
            "addressLocality": "Kreuzberg",
            "postalCode": "10969"
        },
        "location": {
            "type": "Point",
            "coordinates": [13.3903, 52.5075]
        },
        "name": "Checkpoint Markt"
    }
]

Filter context data by querying metadata

This example returns the data of all Store entities with a verified address.

Metadata queries can be made using the mq parameter.

8️⃣ Request:

curl -G -X GET \
    'http://localhost:1026/v2/entities' \
    -d 'type=Store' \
    -d 'mq=address.verified==true' \
    -d 'options=keyValues'

Response:

Because of the use of the options=keyValues, the response consists of JSON only without the attribute type and metadata elements.

[
    {
        "id": "urn:ngsi-ld:Store:001",
        "type": "Store",
        "address": {
            "streetAddress": "Bornholmer Straße 65",
            "addressRegion": "Berlin",
            "addressLocality": "Prenzlauer Berg",
            "postalCode": "10439"
        },
        "location": {
            "type": "Point",
            "coordinates": [13.3986, 52.5547]
        },
        "name": "Bösebrücke Einkauf"
    },
    {
        "id": "urn:ngsi-ld:Store:002",
        "type": "Store",
        "address": {
            "streetAddress": "Friedrichstraße 44",
            "addressRegion": "Berlin",
            "addressLocality": "Kreuzberg",
            "postalCode": "10969"
        },
        "location": {
            "type": "Point",
            "coordinates": [13.3903, 52.5075]
        },
        "name": "Checkpoint Markt"
    }
]

Filter context data by comparing the values of a geo:json attribute

This example return all Stores within 1.5km the Brandenburg Gate in Berlin (52.5162N 13.3777W)

9️⃣ Request:

curl -G -X GET \
  'http://localhost:1026/v2/entities' \
  -d 'type=Store' \
  -d 'georel=near;maxDistance:1500' \
  -d 'geometry=point' \
  -d 'coords=52.5162,13.3777' \
  -d 'options=keyValues'

Response:

Because of the use of the options=keyValues, the response consists of JSON only without the attribute type and metadata elements.

[
    {
        "id": "urn:ngsi-ld:Store:002",
        "type": "Store",
        "address": {
            "streetAddress": "Friedrichstraße 44",
            "addressRegion": "Berlin",
            "addressLocality": "Kreuzberg",
            "postalCode": "10969"
        },
        "location": {
            "type": "Point",
            "coordinates": [13.3903, 52.5075]
        },
        "name": "Checkpoint Markt"
    }
]

Next Steps

Want to learn how to add more complexity to your application by adding advanced features? You can find out by reading the other tutorials in this series:

101. Getting Started
102. Entity Relationships
103. CRUD Operations
104. Context Providers
105. Altering the Context Programmatically
106. Subscribing to Changes in Context

201. Introduction to IoT Sensors
202. Provisioning an IoT Agent
203. IoT over MQTT
204. Using an alternative IoT Agent
205. Creating a Custom IoT Agent
250. Introduction to Fast-RTPS and Micro-RTPS

301. Persisting Context Data using Apache Flume (MongoDB, MySQL, PostgreSQL)
302. Persisting Context Data using Apache NIFI (MongoDB, MySQL, PostgreSQL)
303. Querying Time Series Data (MongoDB)
304. Querying Time Series Data (CrateDB)
305. Big Data Analysis (Flink)

401. Managing Users and Organizations
402. Roles and Permissions
403. Securing Application Access
404. Securing Microservices with a PEP Proxy
405. XACML Rules-based Permissions
406. Administrating XACML via a PAP

501. Creating Application Mashups
503. Introduction to Media Streams
507. Cloud-Edge Computing

601. Introduction to Linked Data
602. Linked Data Relationships and Data Models
603. Traversing Linked Data Programmatically
604. Linked Data Subscriptions and Registrations

The full documentation can be found here.

Iterative Development

The context of the store finder demo is very simple, it could easily be expanded to hold the whole of a stock management system by passing in the current stock count of each store as context data to the Orion Context Broker.

So far, so simple, but consider how this Smart application could be iterated:

Real-time dashboards could be created to monitor the state of the stock across each store using a visualization component. [Wirecloud]
The current layout of both the warehouse and store could be passed to the context broker so the location of the stock could be displayed on a map [Wirecloud]
User Management components [Wilma, AuthZForce, Keyrock] could be added so that only store managers are able to change the price of items
A threshold alert could be raised in the warehouse as the goods are sold to ensure the shelves are not left empty [publish/subscribe function of Orion Context Broker]
Each generated list of items to be loaded from the warehouse could be calculated to maximize the efficiency of replenishment [Complex Event Processing - CEP]
A motion sensor could be added at the entrance to count the number of customers [IDAS]
The motion sensor could ring a bell whenever a customer enters [IDAS]
A series of video cameras could be added to introduce a video feed in each store [Kurento]
The video images could be processed to recognize where customers are standing within a store [Kurento]
By maintaining and processing historical data within the system, footfall and dwell time can be calculated - establishing which areas of the store attract the most interest [connection through Cygnus to Apache Nifi]
Patterns recognizing unusual behaviour could be used to raise an alert to avoid theft [Kurento]
Data on the movement of crowds would be useful for scientific research - data about the state of the store could be published externally. [extensions to CKAN]

Each iteration adds value to the solution through existing components with standard interfaces and therefore minimizes development time.

Name		Name	Last commit message	Last commit date
Latest commit History 212 Commits
.github/workflows		.github/workflows
.env		.env
FIWARE Getting Started.postman_collection.json		FIWARE Getting Started.postman_collection.json
LICENSE		LICENSE
README.es.md		README.es.md
README.ja.md		README.ja.md
README.md		README.md
README.pt.md		README.pt.md
docker-compose.yml		docker-compose.yml
services		services

License

Maghnie/tutorials.Getting-Started

Folders and files

Latest commit

History

Repository files navigation

Contents

Architecture

Prerequisites

Operating System

Docker

Docker Compose (Optional)

Starting the containers

Option 1) Using Docker commands directly

Option 2) Using Docker Compose

Creating your first "Powered by FIWARE" app

Checking the service health

1️⃣ Request:

Response:

Creating Context Data

2️⃣ Request:

3️⃣ Request:

Data Model Guidelines

All terms are defined in American English

Entity type names must start with a Capital letter

Entity IDs should be a URN following NGSI-LD guidelines

Data type names should reuse schema.org data types where possible

Use camel case syntax for attribute names

Location information should be defined using address and location attributes

Use GeoJSON for codifying geospatial properties

Attribute Metadata

Querying Context Data

Obtain entity data by ID

4️⃣ Request:

Response:

Obtain entity data by type

5️⃣ Request:

Response:

Filter context data by comparing the values of an attribute

6️⃣ Request:

Response:

Filter context data by comparing the values of a sub-attribute

7️⃣ Request:

Response:

Filter context data by querying metadata

8️⃣ Request:

Response:

Filter context data by comparing the values of a geo:json attribute

9️⃣ Request:

Response:

Next Steps

Iterative Development

License

About

Resources

License

Stars

Watchers

Forks

Releases

Packages 0

Languages

Location information should be defined using `address` and `location` attributes

Packages