From 6faa7f0b1de4c97ecaf735858e599ede2d1e7f93 Mon Sep 17 00:00:00 2001 From: Sergei Lavrentev <23312691+lavrd@users.noreply.github.com> Date: Sat, 16 Dec 2023 00:07:28 +0400 Subject: [PATCH] Staex IoD Application --- applications/staex-iod.md | 333 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 333 insertions(+) create mode 100644 applications/staex-iod.md diff --git a/applications/staex-iod.md b/applications/staex-iod.md new file mode 100644 index 00000000000..12d431f5e2b --- /dev/null +++ b/applications/staex-iod.md @@ -0,0 +1,333 @@ +# StaexIoD + +- **Team Name:** Staex +- **[Level](https://github.com/w3f/Grants-Program/tree/master#level_slider-levels):** 3 + +## Project Overview :page_facing_up: + +### Overview + +Staex Internet of Data. + +The goal of this project is to create a Web3 IoT data infrastructure with a stable economy. In simple words we want to simplify Web3 onboarding process and bring more IoT device owners to share their useful data with some profit and other people to find and use such data transparently, securely and easily for their life or research. + +We want to bring Web3 data access infrastructure to your ecosystem. As a part of Web3 Foundation ecosystem we want to use Kusama & Shiden parachain for our smart contracts and ecosystem tokens for project economy. + +Our team is interested in creating this project because Staex are working on several solutions for IoT devices, so we are fully interesting to propose such topics to community and improve IoT field at all with new solutions. + +### Project Details + +#### Architecture + +![architecture](https://raw.githubusercontent.com/lavrd/cdn/main/image.png) + +#### IoT devices indexing + +To give users possibility to find particular type of data or particular IoT device we need to index them somehow. Indexing part we want to implement through DIDs and Subsquid project. + +So we will implement ink! smart contract which will be deployed on Shiden network while testnet and on Shiden network after it. This smart contract will have several methods: + +- Create DID +- Add attribute +- Get attribute +- Update attribute +- Delete attribute + +All these methods will produce smart contract events which can be indexed by Subsquid later. + +To index and retrieve IoT devices (DID) we want to use Subsquid project and its API as it index all transactions and events from blockchain network and we can query them using GraphQL API from our UI. User can use public indexer or start its own. Subsquid already has an integration with Kusama, Shiden. + +To make process of setup or update DID for IoT device easier, we will implement such logic in our provisioner. So user will need just to set some fields in config file and provisioner will sync DID attributes with network through smart contracts calls. + +Attributes which we want to store in DIDs: + +- Type of data - type of data which IoT device produces, it can be temperature, humidity or something like CCTV camera output, etc. +- Location - it can be coordinates or name of the place. +- Public key - device public key. We will need it later to create shared secret or know exact files location in IPFS. +- Price to access data per KB. +- Price to pin data per hour. +- Device can include some other attributes but it is optional. + +After that we will implement a UI where users can see devices and find necessary one through filters. UI will be quite simple: header/footer + a table with device list, each row will have an information from DID. Plus we need an input or select field for filters. + +#### ACL + +Access control layer is here to restrict data usage from users which didn't pay for a data. + +And here is a flow: + +1. User puts some tokens to smart contract for exact owner and point which type of data it needs. +2. Provisioner on IoT device sees an event from smart contract that there is new data access request. +3. If provisioner can produce such data. +4. Provisioner gets tokens from smart contract. +5. Provisioner starts to distribute data for this user. + +##### Data access smart contract + +To implement flow below we will implement a kind of multisig ink! smart contract with three methods: + +- Request - user requests for particular data access and sends required amount of tokens. +- Accept - device accepts request and gets required amount of tokens. +- Reject - device rejects request and tokens will be returned to user. + +Device can reject request because of: + +- Requested data is not available at the moment. +- Device has high amount of request (highload). + +This contract will use DID smart contract to check that user sent required amount of tokens otherwise reject automatically. + +##### Data distribution + +To distribute data for exact user we want to use IPFS and it is fully open source, distributed and IoT devices don't need to have public IP. + +To make device root folder static in IPFS network we will use IPNS. With IPNS user which requested for data can look for exact place in IPFS to find it. So IPFS structure will be following: + +```shell +/ipfs/// +``` + +With such structure every user can know where requested data is located. + +When provisioner gets tokens from smart contract it will start distributing process. To prevent data reading from IPFS by users which didn't pay for it, provisioner will encrypt data before upload. To generate encrypt key we will use Diffie–Hellman key exchange. With this algorithm we can generate encrypt/decrypt key without transferring it by the network. So device private key + user public key and device public key + user private key can generate exact the same shared secret key which provisioner will use for encryption. + +After getting data from database and encrypting it provisioner will save and pin data in IPFS and notify user through IPFS pub/sub protocol. + +##### Data accessing + +To access decrypted data automatically user can start provisioner on their own laptop or server. Provisioner will start local IPFS node and subscribe for IPFS events and start sync files from IPFS to local database. Also it will generate shared secret key and decrypt all data while sync process. + +Also provisioner will start simple server to access data from UI through HTTP. UI will be simple and user can see a table with synced data. Each row will have some metadata like IoT device public key, data created time and data itself. Such UI also will be available on device to see produced data. + +##### Data access prolongation + +For example data cost 1 token per 1KB. If device saved to IPFS 1KB of data it will stop data distribution. If user doesn't want such data access prevents it should put more tokens on smart contract on data request. When provisioner almost done with 1KB it will try to get more tokens from smart contract and in case of success continue to distribute data. + +##### Data access expiration + +Through data access smart contract user can request data pinning for exact time interval. So while this time interval provisioner will keep data inside IPFS and will notify user in case data is expired through IPFS pub/sub. + +#### Provisioner + +Provisioner is a background daemon which will be written in Rust. It is responsible for: + +- Manage applications lifecycle through Docker containers. +- Sync local DIDs attributes from config file with smart contract. +- Manage local IPFS node lifecycle and distribute data through it. + +Also provisioner will have a HTTP server to work with user interfaces through it. For example provisioner will have a UI to see running applications, data and data access subscribers. It will be simple UI with a table for each type of data and a table with data consumers. + +SQLite database structure: + +- Table "data_" - in this table we will store data from application. + - id - serial. + - type - data type (temp, speed, etc.). + - data - bytes. + - created_at - timestamp. +- Table "consumers" - in this table we will store data access requests and consumed amount if data by users. + - id - serial. + - public_key - consumer public key. + - requested - requested amount in KB. + - consumed - consumed amount in KB. + - created_at - timestamp. +- Table "sync" - to know data sync process with IPFS. + - id - serial. + - consumer_id - consumer.id. + - data_id - data.id. + - synced_at - timestamp. + - expired_at - timestamp, when data should be deleted from IPFS. + - created_at - timestamp. + +To make it possible for parallel writes using SQLite from applications, provisioner will make migrations for each application while starting it and we will setup SQLite to use WAL mode. + +Provisioner will have at least one running application - IPFS node. + +Also we will implement a command line interface to start/stop application by provisioner. We want to use for it CLI instead of UI because it is the same experience as with kubectl for Kubernetes. It will be easier to manage application lifecycle with just some commands. + +#### Applications + +Applications are small programs to read data from different types of sensors. Through config file device owner can simply start them and see how data from sensors are securely distributed to users on request. + +In the current version applications are Docker containers because it is simple to build them for different IoT devices and provisioner will be responsible for their lifecycle as an orchestrator. Also with Docker containers applications can be written in any language. + +Applications should write to local SQLite database to give and access to data from sensors to provisioner for their future work. data exchange + +#### Technologies + +- IPFS - for decentralized data distribution and notifications. +- Rust - to effectively implement provisioner. +- Docker - to ship applications and manage their lifecycle. +- ink! - smart contracts as a transparent and honest way to implement data access economy. +- Vue - for fast and comfortable implementation of user interfaces. +- SQLite - to share data between applications, IPFS and provisioner. +- Subsquid - to index and retrieve on-chain data. + +### Ecosystem Fit + +Help us locate your project in the Polkadot/Substrate/Kusama landscape and what problems it tries to solve by answering each of these questions: + +- Where and how does your project fit into the ecosystem? - our project is fit into the ecosystem as distributed data access infrastructure. +- Who is your target audience? - our target audience is IoT device owners and data researches. Moreover data like real-time temperature from particular place can be interesting for a lot of people and other type data too. +- What need(s) does your project meet? - our project meets easier, secure and honest data access. +- Are there any other projects similar to yours in the Substrate / Polkadot / Kusama ecosystem? - we didn't find similar projects. + +## Team :busts_in_silhouette: + +### Team members + +- **Alexandra Mikityuk, CEO.** PhD in Computer Science. Previously co-founder of IT companies, startup consultant, 17 years in the software industry. +- **Paksy Plackis-Cheng, CSO.** Award-winning entrepreneur, 20 years executive management experience in (deep) tech startups and renowned global corporations. IPO and M&A experience. Strategic partnerships, BizDev, GTM. +- **Ivan Gankevich, Product lead / system architect.** PhD in computer science. Previously a researcher in the field of distributed systems with 10 years of experience. +- **Maksim Sukhotin, Senior software engineer.** 13 Years Experience in various corporate projects as senior software engineer. Specialization in industrial process control and automation. +- **Sergei Lavrentev, Senior software engineer.** M.Sc. in Computer Science and 6 years of experience in software development in different areas. + +### Contact + +- **Contact Name:** Sergei Lavrentev +- **Contact Email:** sergei@staex.io +- **Website:** https://staex.io + +### Legal Structure + +- **Registered Address:** Staex GmbH, c/o Unicorn, Am Neuen Markt 9 e-f, 14467 Potsdam, HRB 36981 P +- **Registered Legal Entity:** Staex GmbH + +### Team's experience + +Our team has huge experience in IoT infrastructure field. As a part of projects from T-Labs (Deutsche Telekom) and Staex, we were able to implement orchestrators for Blockchain networks, distributed networks and currently VPN with IoT focus. https://staex.io/product here you can find more about our benchmarks and we are able to implement this solution. + +- https://staex.io/product/features/low-data-usage +- https://staex.io/product/features/high-performance + +Our solution at Staex is written in Rust. + +Also to prove our team experience we want to mention that we have some patents: https://staex.io/patent. + +### Team Code Repos + +Unfortunately at the moment we don't have public repositories. + +Please also provide the GitHub accounts of all team members. If they contain no activity, references to projects hosted elsewhere or live are also fine. + +- https://github.com/igankevich +- https://github.com/lavrd + +### Team LinkedIn Profiles (if available) + +- https://www.linkedin.com/in/dr-alexandra-mikityuk/ +- https://www.linkedin.com/in/paksy/ +- https://www.linkedin.com/in/ivan-gankevich/ +- https://www.linkedin.com/in/max-sukhotin/ +- https://www.linkedin.com/in/lavrdx/ + +## Development Status :open_book: + +We don't start project yet. But we did a research about possible implementation which is described in this document. + +## Development Roadmap :nut_and_bolt: + +### Overview + +- **Total Estimated Duration:** 4 months +- **Full-Time Equivalent (FTE):** 2 +- **Total Costs:** 52,000 USD + +### Milestone 1 — IoT device indexing + +- **Estimated duration:** 1 month +- **FTE:** 2 +- **Costs:** 13,000 USD + +| Number | Deliverable | Specification | +| -----: | ----------- | ------------- | +| **0a.** | License | MIT | +| **0b.** | Documentation | 1. How to use DID smart contract.
2. How to start and setup Subsquid node.
3. How to configure provisioner to sync DID with chain.
4. Attribute definitions and their meaning.
5. How to start UI and use it.
6. How to start local Substrate based node for completely local setup.
7. How to use ink! for our local development.
8. General description of the first milestone and why.
9. Current architecture diagram.
10. Instruction how to test with local development.
11. How to start end-to-end tests. | +| **0c.** | Testing and Testing Guide | ink! smart contract will be covered by unit tests. Provisioner DID logic will be covered through Rust end-to-end tests. | +| **0d.** | Docker | We will provide docker compose to setup local development. | +| **1.** | DID smart contract | DID smart contract implementation. | +| **2.** | Provisioner | Provisioner which can sync DIDs with chain. | +| **3.** | Subsquid integration | Setup Subsquid node to listen on-chain events. | +| **4.** | UI | UI to get data from Subsquid and show indexed IoT devices. | + +### Milestone 2 — Provisioner & Applications + +- **Estimated Duration:** 1 month +- **FTE:** 2 +- **Costs:** 13,000 USD + +| Number | Deliverable | Specification | +| -----: | ----------- | ------------- | +| **0a.** | License | MIT | +| **0b.** | Documentation | 1. How to start and use provisioner CLI/UI.
2. How to start end-to-end tests.
3. How to write new application.
4. How data exchange process works. | +| **0c.** | Testing and Testing Guide | 1. End-to-end tests in Rust for lifecycle management.
2. End-to-end tests in Rust for data exchange.
3. End-to-end tests in Rust to check IPFS application integration.
4. Provisioner OpenAPI documentation. | +| **0d.** | Docker | We will provide docker compose to setup local development. | +| **1.** | Applications lifecycle management | Source code which is responsible for managing Docker containers lifecycle. | +| **2.** | IPFS application | Application with IPFS node which can be run by provisioner. | +| **3.** | Example application | Application which simulate reading from IoT device sensor. | +| **4.** | Data exchange | Provisioner <-> Application data exchange through database. | +| **5.** | Provisioner CLI | CLI to start/stop applications. | +| **6.** | Provisioner UI | See sensors data and data consuming by users. | + +### Milestone 3 — ACL 1 + +- **Estimated Duration:** 1 month +- **FTE:** 2 +- **Costs:** 13,000 USD + +| Number | Deliverable | Specification | +| -----: | ----------- | ------------- | +| **0a.** | License | MIT | +| **0b.** | Documentation | 1. How to use data access smart contract.
2. How data distribution works.
3. How data accessing works. | +| **0c.** | Testing and Testing Guide | 1. Data access smart contract unit tests.
2. Data distribution and access end-to-end tests in Rust. | +| **0d.** | Docker | We will provide docker compose to setup local development. | +| **1.** | Data access smart contract | Smart contract to implement data access requests and tokens flow. | +| **2.** | Smart contract integration | Use data access smart contract from provisioner. | +| **3.** | Data distribution | Provisioner syncs local database with IPFS with data encryption for particular user. | +| **4.** | Data accessing | Provisioner syncs local database with IPFS with data decryption from particular device. | + +### Milestone 4 — ACL 2 + +- **Estimated Duration:** 1 month +- **FTE:** 2 +- **Costs:** 13,000 USD + +| Number | Deliverable | Specification | +| -----: | ----------- | ------------- | +| **0a.** | License | MIT | +| **0b.** | Documentation | 1. Describe new features of our data accessing smart contract.
2. Describe how prolongation and expiration work.
3. Describe how to use our new user interface features. | +| **0c.** | Testing and Testing Guide | 1. End-to-end test in Rust for prolongation and expiration cases.
2. Unit tests for new features in smart contract. | +| **0d.** | Docker | We will provide docker compose to setup local development. | +| 0e. | Article | We will write an article that explains the work done as part of the grant. We will publish this article to our dev.to organization and to our website and blog (https://blog.staex.io). Also we want to post non-technical article on LinkedIn. | +| **1.** | Data access smart contract | Improve data access smart contract to manage data expiration. Improve smart contract to keep user tokens for prolongation. | +| **2.** | Data access prolongation | Improve provisioner to see tokens on smart contract and prolong data access. | +| **3.** | Data access expiration | Improve provisioner to remove data from IPFS based on data expiration. | +| **4.** | Accessing data interface | Improve database structure and UI to see decrypted data from device on user machine. | + +## Future Plans + +We already have a lot of improvement and plans for feature: + +1. At the moment we will implement history or near-realtime data access. But sometimes devices want to distribute their data in realtime. So it is a huge task and we really want to implement it. It will be implemented with MQTT and some kind of mesh network solution which provide fast and secure access to IoT device data stream (possible usage of StaexMCC?). Btw we already have some docs how to implement it. +2. It is crucial to implement some kind of oracle to check that data was delivered to user and this data is not a garbage. +3. Some kind of data indexing in a folder so user can apply some filters or make data sorting. +4. Marketplace - some kind of public marketplace on IPFS to distribute applications and provide reward for application developers. +5. Sometimes it is not possible to start so many services on single IoT device or store a lof of data, so we want to give devices possibility to use remote IPFS for data distribution or data pinning. +6. Support small boards like STM32/ESP32. +7. Provide user possibility to request historical data, for example for the past year. +8. Auto-detecting sensors on IoT device and start applications according to it. +9. New type of data consuming. In this grant we will implement counting by amount of data, but it is also possible to request for a data not in "bytes" but in events count. +10. More user friendly price system for mass adoption. IoT device can ask for their data in some token, for example, DOT, but on the website we can ask for data access, for example, USDT. So we can automatically make an exchange or teleport tokens to parachain to make it easier to sell or buy data access. + +For project promotions we want to use our community from LinkedIn and partners. As a part of our content marketing we will put some articles to our own blog on our website and through dev.to organisation. Also we are almost started our podcast. + +## Referral Program (optional) :moneybag: + +You can find more information about the program [here](../README.md#moneybag-referral-program). + +- **Referrer:** [David Hawig](https://github.com/Noc2) + +## Additional Information :heavy_plus_sign: + +**How did you hear about the Grants Program?** Web3 Foundation Website / Medium / Twitter / Element / Announcement by another team / personal recommendation / etc. + +We found this program by suggestion after winning in [Web3 Foundation hackathon](https://staex.io/paris-blockchain-week-2023). [External article](https://www.essec.edu/en/news/unlocking-future-web3-key-highlights-after-paris-blockchain-week-exclusive-talk-michael-amar-chairman-pbw/).