Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added a sentence about supported database technologies #64

Merged
merged 3 commits into from
Jul 22, 2024
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions content/03.06.03.Zendro.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
#### GraphQL Data-warehouse

<!-- Asis Hallab -->
Using the [Zendro](https://zendro-dev.github.io) set of automatic software code generators, a fully functional, efficient, and cloud-capable BrAPI data-warehouse has been created for the current version of the BrAPI data models. Unlike most BrAPI compliant data sources, this data-warehouse supports a GraphQL API rather than a RESTful API. This API provides secure access to data read and write functions for all BrAPI data models. It provides create, read, update, and delete (CRUD) functions that are standardized and accept the same parameters for all data models.
Using the [Zendro](https://zendro-dev.github.io) set of automatic software code generators, a fully functional, efficient, and cloud-capable BrAPI data-warehouse has been created for the current version of the BrAPI data models. Unlike most BrAPI compliant data sources, this data-warehouse supports a GraphQL API rather than a RESTful API. This API provides secure access to data read and write functions for all BrAPI data models. It provides create, read, update, and delete (CRUD) functions that are standardized and accept the same parameters for all data models. Zendro supports a large number of database systems which and can even be mixed, among these support exists for relational (e.g. PostGres), document based (e.g. MongoDB), and graph based databases (Neo4J), as well as wide column stores (e.g. Cassandra) and file clouds (e.g. Amazon AWS), additionally the data-lake technologies [Trino](https://trino.io/) and [Presto](https://prestodb.io/) can be used.

The GraphQL server is particularly rich in features. Records are paginated using the highly efficient cursor based pagination model as proposed in the GraphQL standard. Logical filters allow for exhaustive search queries, whose structure is highly intuitive and based around logical triplets. A large collection of operators is available and triplets can be combined to logical search trees using "and" or "or" operators. Searches can be extended over relationships between data models, thus enabling a user to query the warehouse for exactly the required data. Access security is implemented with the OAuth2 user authentication standard [@https://datatracker.ietf.org/doc/html/rfc6749]. Authorization is based on user roles and can be configured differently for each single data model read or write function. The generated graphical interface allows for the integration of interactive scientific plots and analysis tools written in JavaScript or WebAssembly.

The Zendro data-warehouse is capable of forming an efficient cloud of data servers. This is achieved simply by linking other Zendro based warehouses that expose the same GraphQL API to the same data models, or a subset of data models. Any network of such Zendro GraphQL servers can be set up using this configuration approach. The code generated then exposes full access to all data records stored on any node of the network, while maintaining full security control at each node. Importantly, the warehouses are programmed in such a way that any number of data servers can be joined without loss of efficiency.

<!-- Asis Hallab: To Do: (i) provide the link to an example running data warehouse, and (ii) include and describe example scientific plots. -->

The Zendro data-warehouse is capable of forming an efficient cloud of data servers. This is achieved simply by linking other Zendro based warehouses that expose the same GraphQL API to the same data models, or a subset of data models. Any network of such Zendro GraphQL servers can be set up using this configuration approach. The code generated then exposes full access to all data records stored on any node of the network, while maintaining full security control at each node. Importantly, the warehouses are programmed in such a way that any number of data servers can be joined without loss of efficiency.
We used parts of the public [CassavaBase](https://www.cassavabase.org/) data [@doi:10.1093/nar/gku1195] to create and populate a fully BrAPI compliant example data-warehouse based on Zendro. The warehouse is publically available [@https://brapi-gui.zendro-dev.org] and offers full read access both in the graphical user interface as well as through the GraphQL API. Three interactive scientific example plots comprise a boxplot comparing Cassava harvest indices measured for four different experiments. An interactive raincloud plot provides an alternative visualization of the same data. Finally, a scatterplot shows how Cassava fresh root yield and plant height are correlated based on data from a single study.
Loading