-
Notifications
You must be signed in to change notification settings - Fork 75
Data Structure Guidelines
This page illustrates how to structure data for OriginTrail protocol to utilize data sharing and connectivity functionalities of the Alpha version of the OriginTrail Distributed Network (ODN). It corresponds to attachments and examples that are provided with this file and the documentation on GitHub. The defined structure was developed on the best practices observed over the last several years of experience in implementing transparency solutions within the food supply chain industry, and is based on GS1 standards.
OriginTrail is a protocol that enables exchange of standardized data among disparate IT systems in multi organizational environments in a tamper proof way. Therefore, each participant of the exchange should provide their data in a common and standardized format. To utilize the OriginTrail protocol to showcase its full potential, the communication from ERP systems of at least two entities within a supply chain towards ODN needs to be established. The communication is done periodically by sending standardized XML files to the OriginTrail network node, the structure of which will be further explained in this document.
The upload of data in the XML format to the OriginTrail protocol node is performed via the web-service endpoint. The process of extracting data from the ERP system, including its periodic forwarding to the OriginTrail node API endpoint is out of the scope of this document. This document focuses on the XML data structure that the Alpha network can import and utilize.
You can view relevant GS1 EPCIS standards on following link.
- Collect the data samples that you want to store on ODN according to defined use case
- Choose data structure and download sample files with XSD scheme
- Connect XSD scheme and sample file in some advanced XML editing software
- Modify sample file according to your data architecture and validate changes against XSD file.
- Map your data structure with sample file.
- Generate file from your ERP and test the data against XSD file.
- Send file to ODN via WebService.
OriginTrail is using a GS1 standards for XML data structure. The standards were implemented in several iterations so we have two basic types:
- full GS1 compliant data structure
- GS1 based data structure (older version)
This file structure is based on GS1 (EPCIS and CBV) guidelines and it is fully compliant to XSD schemes provided by GS1. Syntax and XML node structure fully corresponds to GS1 standards and provided XSD schemes. OriginTrail has introduced namespace (urn:ot:* ) for custom identifiers. If there are namespaces covered by GS1 standard (like for Business Location entity) then they will have advantage over custom namespaces.
The OriginTrail XML file contains three main logical parts document data, master data and visibility events data. All parts of the file must be according to GS1 XSD scheme.
The example XML file, as well as the XSD scheme that can be found here. You can refer to GS1 documentation for further guidelines for data structuring.
We strongly advise use of advanced XML editing software that will verify if your data structure corresponds to XSD scheme that is proposed by GS1 if you want to have full compliance of the data structure.
EPCIS guideline suggests Standard Business Document Header SBDH standard for description of the document data. This part of data is in EPCIS Header part of the file. It has basic information about file (sender,receiver,ID,purpose...). Although OriginTrail is the receiver of the file and it can be named as receiver (SBDH allows defining multiple receivers) it is not necessary to include this. Receiver is some entity involved in a business process, not in the data processing.
EPCIS standard describes 4 ways to process Master data. OriginTrail is going to support the most common way - including Master data in the Header of an EPCIS XML document. Master data will be processed separately from other data. There is no need to include master data in every file. Only data that has not been sent to ODN until that moment should be provided. If there is already the same version of master data on ODN, it will be omitted and not processed further. But if there is visibility event data that is related to some master data that is not on ODN, there can be query problems in graph database (the file is going to be processed on ODN anyway).
More information about master data namespaces and entities can be found here.
Main focus of EPCIS standard is formalizing description of event data that are generated by activities within supply chain. OriginTrail is focusing on ObjectEvent and TransformationEvent that are thoroughly described in the standard (although other event types are also supported). We strongly advise to read GS1 EPCIS implementation guideline and to evaluate our example files.
This is first and simplified data structure that was utilized in OriginTrail protocol. It is based on GS1 standards, but it is not following proposed standard in some key areas. The main goal of this data structure was to simplify XML for certain pilots that were conducted in order to establish some main data entities in more transparent way and to speed up implementation process. This data structure lacks flexibility for further development and GS1 syntax is not fully appropriate. XSD schemes provided are modified version of GS1 EPCIS schemes. Namespaces are not used in this data structure. Introduction of new data entities requires modification of XML importer code.
The OriginTrail XML file contains three main logical parts master data, transaction and visibility events data. The system is flexible, as it doesn’t require all these data sets to be in one single file.
The example XML file, as well as the XSD scheme that can be found here contain most of the explanations in comments. Data guidelines specified in the provided XML are modular, extensible and layered and will likely be updated according to requirements of each specific use case. Please see both files for more details.
The XML consists of 3 main logical parts.
The first part of the file contains the ID of the data provider, which is the ParticipantID. This value should be unique for each participant of the chain. During the test phase you can obtain your unique ID here.
Master data is the core data that is essential to operations in a specific business or business unit. It represents the business objects which are handled and shared across the enterprise. In our current XML structure, the Master Data entities are:
- ParticipantsList and Participant – Description of all entities (participants) in the supply chain
- BusinessLocationsList and BusinessLocation - describe all physical or system locations where business processes are executed
- ObjectsList and Object – entities that are subjects of business processes (items or goods being described)
Transaction data describes interactions between entities described with master data by the data provider. OriginTrail distinguishes two types transaction data:
-
Internal transactions are related to processes of object movements or transformations (production, repackaging etc) within the scope of one supply chain participants business location. For example, this could be production or assembly transactions that result in product output for further production or sale (repackaging, labeling etc).
-
External transactions are related to processes between different supply chain participants (sales / purchases, transport). They represent processes where the jurisdiction or ownership of the objects gets changed in the supply chain.
Each transaction should have a unique ID that represents the primary key within the ERP transaction table from the data source. The transaction data implies that the provider of that data is one of the active participants in the transaction process.
Visibility event data covers details about physical or digital activity in the supply chain of an object batch (explained by the master and transaction data above). It references information collected by either sensors or other external entities (i.e. lab test results) which provide more details about a specific object batch in a specific point in time or time span.
To start integration with OriginTrail, a periodic upload of the appropriately structured XML file (according to the XSD scheme above) should be set up. Please check https://github.com/OriginTrail/ot-node/wiki/Installation-Instructions for further details.