Welcome to this Github Repo azureiotpyvision

In this repo you will find a Python code that can used in your Raspberry Pi to build an Intelligent Vision solution with Azure IoT and Azure Custom Vision Cognitive Services to send telemetry to Azure Iothub

Along with the python code, in this repo you will find the Hardware settings and the Azure Components been used to successfully execute the application. At this point, this is no a step by step guide, the main point is for you to get and understand the Python code used to develop the solution.

Guide information

Reference architecture

Hardware - Vision components

• Raspberry Pi model 4
• Grove Base Hat for Raspberry Pi
• Grove - Temperature & Humidity Sensor (DHT11)
• Raspberry Pi High Quality Camera
• Arducam CS-Mount Lens for Raspberry Pi HQ Camera

Local Environment requirements

• Internet connectivity
• Visual Studio Code
• Python version 3
• Hypervisor - In my case I was using my own laptop with Windows 11 and I enabled Hyper-V
• A virtual machine with IoT Edge
  • A docker container with Azure Custom Vision

Azure requirements

• An Azure subscription with administrative access
• Azure Iothub
• Azure Custom Vision

Environment set up - You might use the following links in case you want to recreate the solution

• Connection and configuration of the Raspberry Pi
• Connect the Camera to the Raspberry Pi
  • Text Instructions
  • Video Instructions
• Create an Azure Iothub service
• Create a Azure Custom Vision service and train the model
  • You can train Azure Custom Vision to interpret whatever figure you want. In my example I am training the service to interpret a "lighter".
• Create an Iotedge device and deploy a Custom Vision model container
  • In my environment, the Iot Edge was created in a Linux VM inside Hyper-V

Application Code

• The code is located in "code/raspbpi-customvision-app.py"

Python Libraries

• picamera
• seeed_dht
• requests
• matplotlib
• Pillow
• Numpy
• azure-iot-device

Application Notes

• The application must run inside the Raspberry Pi
• The Raspberry Pi must install previous Python Libraries
• The Raspberry Pi must have Python3

Application main functions and methods

• Main Function:

  • Camera Initialization.
  • Temperature Sensor Initialization.
  • Connection establishment to Azure Iothub.
  • Calls multiple method to capture image in case the environment method is above 32 Celsius.
  • It sends a JSON object with the temperature and custom vision information to Azure IoThub.

• CaptureImage Function:

  • Takes a picture using the camera.

• CustomVisionQuery Function:

  • Sends the picture taken by the CaptureImage function to the Custom Vision Container in Iotedge (this IoT Edge is reacheable using a private IP).
  • Saves the JSON response from the Custom Vision container to a variable.

• DetectedObjects Function:

  • Creates an image with lines around each detected object from the Custom Vision container with a probability above 70%.
  • The position of the lines are determined by bounding box coordinates from the Custom Vision Container.
  • With the detect object coordinates, it creates a new image with lines around each detected object.
  • The image is saved locally in the Raspberry Pi.

• MergeVisionTemperature Function:

  • Creates a Dictonary variable IF temperature is higher than 32Celsius and Custom Vision Probability result is higher than 70%.
  • The Dictonary variable is used to create a JSON object

If you have any question, suggestion or comment about this repo or code, feel free to post it in this discussion section. I will make sure to address all your concerns.