Skip to content

Commit

Permalink
minor updates and G-Crunch re-org
Browse files Browse the repository at this point in the history
  • Loading branch information
BrapiCoordinatorSelby committed Aug 1, 2024
1 parent e787e4c commit 178c9b9
Show file tree
Hide file tree
Showing 8 changed files with 17 additions and 12 deletions.
6 changes: 3 additions & 3 deletions content/02.introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,15 +15,15 @@ Notes:

To address consequences of climate change and population growth, plant and animal breeding needs to become more efficient and data driven to ensure a healthy, resilient, and sustainable agricultural production system. Modern breeding techniques require large amounts of high quality data to be effective, requiring digital methods for data collection, management, and analysis. They rely on several research disciplines, including plant phenomics, genetics, genomics, and agronomy, involving research institutes, genebanks, and breeding companies. Interoperability between research software tools, systems, and databases can substantially increase the efficiency of a breeding program. The ability to efficiently share data means access to larger and more complete datasets, enabling more accurate computational models, more accurate predictions, and improved selections.

The Breeding API (BrAPI) project is an effort to enable interoperability among breeding tools, systems, and databases. BrAPI is a standardized Representational State Transfer (REST), web service, Application Programming Interface (API), specification for breeding, genetics, phenomics and related agricultural data. [@doi:10.1093/bioinformatics/btz190] By using the BrAPI standard, breeding software can more easily become interoperable, allowing groups to more easily share data and software tools.
The [Breeding API (BrAPI)](https://brapi.org) project is an effort to enable interoperability among breeding tools, systems, and databases. BrAPI is a standardized Representational State Transfer (REST), web service, Application Programming Interface (API), specification for breeding, genetics, phenomics and related agricultural data. [@doi:10.1093/bioinformatics/btz190] By using the BrAPI standard, breeding software can more easily become interoperable, allowing groups to more easily share data and software tools.

Since its first publication [@doi:10.1093/bioinformatics/btz190], BrAPI has seen a significant increase in community services, compatible tools, and participating organizations. The community has organized numerous hackathons to evolve the specifications, resulting in continuous improvements and enhancements. This report includes a short technical description of the standard and a showcase of the applications, services, and tools available in the BrAPI community. BrAPI has become an essential part of the digital infrastructure for breeding, genetics and phenomics applications and related agricultural projects. It is the intention of this manuscript to demonstrate the value of BrAPI to the wider scientific community as an effective and efficient means to collaborate and share resources.

### How it works

An API is a technical connection between two pieces of software. Just as a Graphical User Interface (GUI) or a Command Line Interface (CLI) allows a human user to interact with a piece of software, an API allows one software application to interact with another. A GUI or CLI might allow a user to input data, read data, and start processes within an application. An API allows one piece of software (sometimes called a client, user agent, or service consumer) to programmatically input data, read data, and start process within another piece of software (sometimes called a server or service provider).

A REST style web service is a type of API commonly used in today's web infrastructure. REST is a technical architecture that describes the stateless transmission of data between applications. Typically, RESTful web service APIs are implemented using the standard HTTP protocol that most of the modern internet is built upon. These implementations generally use JavaScript Object Notation (JSON) to represent the data being transferred. Both HTTP and JSON are programming language agnostic, very stable, and very flexible. This means BrAPI can be implemented in almost any piece of software, and can solve a wide range of use cases.
A REST style web service is a type of API commonly used in today's web infrastructure. REST is a technical architecture that describes the stateless transmission of data between applications. Typically, RESTful web service APIs are implemented using the standard HTTP protocol that most of the modern internet is built upon [@doi:10.1145/514183.514185]. These implementations generally use JavaScript Object Notation (JSON) to represent the data being transferred. Both HTTP and JSON are programming language agnostic, very stable, and very flexible. This means BrAPI can be implemented in almost any piece of software, and can solve a wide range of use cases.

Data repositories and service providers can choose to represent their data as a BrAPI compatible API. By mapping their internal data structures to the standard models, data repositories can easily expose data to the outside world. Similarly, they can accept new data from external sources and automatically map the new data into an existing database. Client application developers can take advantage of this standardization by building tools and connectors that integrate with all BrAPI compatible data repositories. Visualization, reporting, analytics, data collection, and quality control tools can be built once and shared with other organizations following the standards. As the number of BrAPI compatible databases, tools, and organizations grows, so does the value added by implementing the standard into a given application.

Expand All @@ -39,4 +39,4 @@ As the specification has matured, so have the tools, services, and libraries ava

The international BrAPI Community consists of software developers, biologists, and other scientists working on BrAPI related projects and data sources. This community is what sustains the BrAPI project, builds implementations, maintains development tools, and provides input to enhance the specification. As the project has grown, so has the community. The BrAPI project started in June 2014 with less than ten people coming together to discuss the idea. Over the next ten years, the community has grown to between 200 and 250 members.

The BrAPI Hackathons are a major staple of the BrAPI community. Twice a year, the community gathers to discuss the specification and collaborate on BrAPI related projects. This time is very valuable to the community; for some organizations, the hackathon is the only time during the year when they can collaboratively work on anything related to BrAPI. During the COVID-19 pandemic, virtual hackathons took the place of in-person events. While the virtual hackathons do not provide the same level of face-to-face time that is crucial to collaborative work, they did allow for more attendees to gather and share their opinions. Going forward, the community leadership has decided to have one in-person hackathon and one virtual hackathon each year, to balance the advantages of both.
The BrAPI Hackathons are a major staple of the BrAPI community [@doi:10.1101/gr.228460.117]. Twice a year, the community gathers to discuss the specification and collaborate on BrAPI related projects. This time is very valuable to the community; for some organizations, the hackathon is the only time during the year when they can collaboratively work on anything related to BrAPI. During the COVID-19 pandemic, virtual hackathons took the place of in-person events. While the virtual hackathons do not provide the same level of face-to-face time that is crucial to collaborative work, they did allow for more attendees to gather and share their opinions. Going forward, the community leadership has decided to have one in-person hackathon and one virtual hackathon each year, to balance the advantages of both.
2 changes: 1 addition & 1 deletion content/03.00.HEADER.Success.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
## Results
<!-- success stories highlighting BrAPI usefulness in breeding cycle. Perhaps reference the original BrAPI paper where possible use cases were proposed. -->

Below are a number of short success stories from the BrAPI community. These tools, applications, and infrastructure projects serve as another indicator of community growth and success over the past 5-10 years. These stories clearly illustrate all the different ways the BrAPI Standard can be used productively and in practice. Figure {@fig:apps} contains a summary of the tools described below.
Below are a number of short success stories from the BrAPI community. These tools, applications, and infrastructure projects serve as another indicator of community growth and success over the past 5-10 years. These stories clearly illustrate all the different ways the BrAPI standard can be used productively and in practice. Figure {@fig:apps} contains a summary of the tools described below.

![A summary of all the tools described below and the general areas each tool is designed to handle](images/BrAPI_Paper_Applications_Chart.png){#fig:apps width="100%"}

Expand Down
2 changes: 1 addition & 1 deletion content/03.03.02.FLORILEGE.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,6 @@
<!-- Alice B -->
[Florilège](https://florilege.arcad-project.org/) is a web portal designed primarily for the general public to access public plant genetic resources held by Biological Resource Centers across France. Through this portal, users can browse accessions from over 50 plant genera, spread across 19 genebanks. It allows users to view available seeds and plant material, including options for ordering material. Florilège provides a centralized access to the various French collections of plant genetic resources available to the public.

Florilège retrieves accession information from several BrAPI-compliant systems. Key among these are OLGA, a genebank accessions management system, and GnpIS, an INRAE data repository for plant genetic resources, phenomics, and genetics [@doi:10.34133/2019/1671403;@doi:10.1007/978-1-4939-6658-5_5]. Using BrAPI to gather data from these systems reduced development efforts and enabled standardized data retrieval. As a result, BrAPI has become the de facto standard within the French plant genetic resources community for exchanging information. During development, the Florilège team also proposed several enhancements to the BrAPI specifications themselves, such as additional support for Collection objects or improved reference linking, to better accommodate their specific use case.
Florilège retrieves accession information from several BrAPI-compliant systems. Key among these are [OLGA](https://crb-plantes-olga.fr/public/frontend/auth/login), a genebank accessions management system, and [GnpIS](https://urgi.versailles.inra.fr/Tools/GnpIS), an INRAE data repository for plant genetic resources, phenomics, and genetics [@doi:10.34133/2019/1671403;@doi:10.1007/978-1-4939-6658-5_5]. Using BrAPI to gather data from these systems reduced development efforts and enabled standardized data retrieval. As a result, BrAPI has become the de facto standard within the French plant genetic resources community for exchanging information. During development, the Florilège team also proposed several enhancements to the BrAPI specifications themselves, such as additional support for Collection objects or improved reference linking, to better accommodate their specific use case.

<!-- ![Figure Florilege Workflow](images/Schema_Florilege.jpg){#fig:Schema_Florilege width="100%"} -->
6 changes: 2 additions & 4 deletions content/03.05.01.G-Crunch.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
#### G-Crunch

<!-- Josh -->
[G-Crunch](https://github.com/CornellILCI/G-CrunchUI) is an upcoming user-facing tool to make simple, repeatable analysis requests. By using BrAPI calls to phenotypic and genotypic data sources, as well as an API currently implemented in [Analytics Framework](https://github.com/CornellILCI/af-pipeline) similar to the proposed [Analytics NCP](https://brapinewconceptpreview.docs.apiary.io/), the lightweight UI can be used to specify and window incoming data, select specific analysis criteria built into Analytics Framework, and trigger any analytics pipeline that's baked into the specific framework instance, such as sommer and ASREML based pipelines currently in Analytics Framework.
[G-Crunch](https://github.com/CornellILCI/G-CrunchUI) is an upcoming user-facing tool to make simple, repeatable analysis requests. The lightweight UI can be used to specify and filter incoming data, select specific analysis criteria, and trigger any analytics pipeline that is built into the specific framework instance. G-Crunch is currently built on top of the open source [Analytics Framework](https://github.com/CornellILCI/af-pipeline) project, and can run pipelines using tools such as sommer and ASREML. Each piece fo the data and pipeline can be separately specified, which can allow flexibility when running complex analysis. A 'test' analysis can be run on small data sets with small or local analytics engine, then quickly re-direct G-Crunch to a larger dataset and a larger computational framework. This mitigates the complications of moving data around and introducing errors from manually triggering the analysis steps.

Since each piece can be separately specified, this can allow flexibility to run 'test' analyses on small data sets with small or local analytics instances, and quickly re-point G-Crunch to a larger organizational or cloud-based Analytics Framework and/or data set, without having to worry about moving data around or introducing errors in manually triggering the analysis steps.

G-Crunch, as a tool, couldn't feasibly exist without BrAPI. The support of BrAPI interfaces allows G-Crunch to use one unified request method, and adapt to the user's existing network of BrAPI-compliant tools. This lowers the barrier to entry for adoption, and makes analysis pipelines easily repeatable.
G-Crunch relies on BrAPI endpoints to access phenotypic and genotypic data sources, as well as an API currently implemented in the Analytics Framework to start and track processes. G-Crunch, as a tool, couldn't feasibly exist without BrAPI. The support of BrAPI interfaces allows G-Crunch to use one unified request method, and adapt to the user's existing network of BrAPI-compliant tools. This lowers the barrier to entry for adoption, and makes analysis pipelines easily repeatable.
2 changes: 1 addition & 1 deletion content/03.05.05.ShinyBrAPPs.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
#### ShinyBrAPPs

The [ShinyBrAPPs](https://github.com/IntegratedBreedingPlatform/ShinyBrAPPs/) code repository contains a number of useful tools, built using the[R-Shiny](https://shiny.posit.co/) framework and the [BrAPI R](https://github.com/mverouden/brapir-v2) open source library. The R-Shiny framework allows user communities to quickly prototype and produce applications that are finely tailored to their needs, thus improving adoption and daily use of data management systems. An international collaboration of developers from CIRAD and the IBP have been working together as part of the [IAVAO](https://www.iavao.org/) breeders community to develop these ShinyBrAPPs, in support of national breeding programs in western Africa. These applications are typically connected to BMS and/or Gigwa and provide tools for specific use cases. BrAPI compliance offers these systems the opportunity to add functionalities in a modular way through the development of external plugin applications that can quickly fulfill specific needs for this group of breeders and scientists.
The [ShinyBrAPPs](https://github.com/IntegratedBreedingPlatform/ShinyBrAPPs/) code repository contains a number of useful tools, built using the [R-Shiny](https://shiny.posit.co/) framework and the [BrAPI R](https://github.com/mverouden/brapir-v2) open source library. The R-Shiny framework allows user communities to quickly prototype and produce applications that are finely tailored to their needs, thus improving adoption and daily use of data management systems. An international collaboration of developers from CIRAD and the IBP have been working together as part of the [IAVAO](https://www.iavao.org/) breeders community to develop these ShinyBrAPPs, in support of national breeding programs in western Africa. These applications are typically connected to BMS and/or Gigwa and provide tools for specific use cases. BrAPI compliance offers these systems the opportunity to add functionalities in a modular way through the development of external plugin applications that can quickly fulfill specific needs for this group of breeders and scientists.

So far, four applications have been developed covering the fields of trial data quality control, single trial statistical analysis, breeding decision support, and raw genotyping data visual inspection. The "BMS trial data explorer" retrieves data from a single multi-location trial and displays data counts and summary box-plot for all variables measured in different studies. It also provides an interactive distribution plot to easily select observations that require curation and a report of candidate issues that needs to be addressed by the breeder. The "STABrAPP" tool is an application for single trial mixed model analysis. It basically provides a GUI to the [StatGen-STA](https://biometris.github.io/statgenSTA/) R package. The "DSBrAPP" tool is a decision support tool helping breeders to select germplasm according to their various characteristics and save this germplasm list into BMS. Finally, the "[snpclust](https://github.com/jframi/snpclust)" tool enables a user to check and manually correct the clustering of fluorescence based SNP genotyping data.
2 changes: 1 addition & 1 deletion content/03.06.03.Zendro.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,4 @@ The GraphQL server is particularly rich in features. Logical filters allow for e

<!-- Asis Hallab: To Do: (i) provide the link to an example running data warehouse, and (ii) include and describe example scientific plots. -->

An [example data warehouse](https://brapi-gui.zendro-dev.org) is publicly available and offers full read access in the graphical user interface and through the GraphQL API. The example warehouse is populated with public [CassavaBase](https://www.cassavabase.org/) data [@doi:10.1093/nar/gku1195] to create fully BrAPI compliant example based on Zendro. Three interactive scientific example plots are available to explore the data. The first is a boxplot comparing Cassava harvest indices measured for four different experiments. Next, an interactive raincloud plot provides an alternative visualization of the same data. Finally, a scatterplot shows how Cassava fresh root yield and plant height are correlated based on data from a single study.
An [example data warehouse](https://brapi-gui.zendro-dev.org) is publicly available and offers full read access in the graphical user interface and through the GraphQL API. The example warehouse is populated with public [CassavaBase](https://www.cassavabase.org/) data [@doi:10.1093/nar/gku1195] to create fully BrAPI-compliant example based on Zendro. Three interactive scientific example plots are available to explore the data. The first is a boxplot comparing Cassava harvest indices measured for four different experiments. Next, an interactive raincloud plot provides an alternative visualization of the same data. Finally, a scatterplot shows how Cassava fresh root yield and plant height are correlated based on data from a single study.
1 change: 1 addition & 0 deletions content/90.back-matter.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ The authors would like to acknowledge the following funding sources:
+ **Valentin Guignon**: Valentin wrote the section on the BrAPI Mapper and MGIS projects, and has made significant contributions to the BrAPI compatibility of the BrAPI Mapper, MGIS, and the Trait Selector BrAPP projects.
+ **Corina Habito**: Corina wrote the section on the BMS project, and has made significant contributions to the BrAPI compatibility of the BMS project.
+ **Asis Hallab**: Asis wrote the section on the Zendro project, and has made significant contributions to the BrAPI compatibility of the Zendro project.
+ **Jenna Hershberger**: Jenna did a significant amount of proofreading and editing to most sections of the manuscript.
+ **Puthick Hok**: Puthick has made significant contributions to the DArT Sample Submission project.
+ **Lynn Carol Johnson**: Lynn wrote the section on the PHG project, and has made significant contributions to the BrAPI compatibility of the PHG project.
+ **Sook Jung**: Sook wrote the section on the BIMS project, and has made significant contributions to the BrAPI compatibility of the BIMS project.
Expand Down
Loading

0 comments on commit 178c9b9

Please sign in to comment.