Skip to content

Commit

Permalink
Merge remote-tracking branch 'origin/main'
Browse files Browse the repository at this point in the history
  • Loading branch information
asishallab committed Jul 18, 2024
2 parents aa2c9d5 + 5a9cf7e commit c0e0238
Show file tree
Hide file tree
Showing 50 changed files with 224 additions and 148 deletions.
1 change: 1 addition & 0 deletions .github/workflows/manubot.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ jobs:
# Set SPELLCHECK to true/false for whether to check spelling in this action.
# For workflow dispatch jobs, this SPELLCHECK setting will be overridden by the user input.
SPELLCHECK: true
BUILD_DOCX: true
defaults:
run:
shell: bash --login {0}
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,5 +27,6 @@ If you would like to be recognized as a co-author, please add your details to th
+ [Markdown Formatting Cheat Sheet](content/formatting_help)
+ [HTML manuscript](https://plantbreeding.github.io/BrAPI-Manuscript2/)
+ [PDF manuscript](https://plantbreeding.github.io/BrAPI-Manuscript2/manuscript.pdf)
+ [MS DOCX manuscript](https://github.com/plantbreeding/BrAPI-Manuscript2/raw/output/manuscript.docx)
+ [Manubot README](Manubot-README.md)
+ [Manubot Usage Instructions](USAGE.md)
8 changes: 4 additions & 4 deletions content/01.abstract.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
## Abstract {.page_break_before}

Population growth and climate change require extraordinary efforts to increase efficiency in breeding programs around the world. In the last few years, new phenotyping techniques, genomics technologies, and genomic prediction approaches have provided a boost in genetic gain in breeding, but have also created a flood of data that needs careful management to be fully harnessed. Data integration is a significant challenge with multiple types of data being collected and stored by a variety of disparate systems.
The Breeding API (BrAPI) project is an international, grass-roots effort to enable more efficient data management by enabling interoperability among research databases and tools, using a standardized RESTful web service API specification for communicating breeding related data.
This community driven standard is software agnostic and free to be used by anyone interested in breeding data management, including trial, germplasm, phenotypic, and genotyping data management.
This manuscript presents the current version of BrAPI, the substantial growth of the project, and a wide variety of open source breeding research tools with active BrAPI implementations.
Population growth and climate change require extraordinary efforts to increase efficiency in breeding programs around the world. In the last few years, new phenotyping techniques, genomics technologies, and genetic approaches such as genomic prediction have provided a boost in genetic gain in breeding, but have also created a flood of data that needs careful management to be fully harnessed. In particular, data integration is a challenge due to the multiple types of data being handled by a variety of disparate and dispersed systems.
The Breeding API (BrAPI) project is an international, grass-roots effort to enable more efficient data management by enabling interoperability among research databases and tools, using a standardized RESTful web service API specification for exchanging breeding related data.
This community driven standard is software agnostic and free to be used by anyone interested in plant breeding, genetics and agronomy data management, including trial, germplasm, phenotyping, and genotyping data management.
This manuscript presents the substantial growth of the project, a wide variety of open source breeding research tools with active BrAPI implementations, and an overview about the current version of BrAPI.
16 changes: 8 additions & 8 deletions content/02.introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,30 +13,30 @@ Notes:
* Standardization of endpoints across objects - said in a non-tech way
-->

To address consequences of climate change and population growth, plant and animal breeding needs to become more efficient and data driven to ensure a healthy, resilient, and sustainable agricultural production system. Modern breeding techniques require large amounts of high quality data to be effective, requiring digital methods for data collection, management, and analysis. Interoperability between breeding software tools, systems, and databases can substantially increase the efficiency of a breeding program. The ability to efficiently share data means access to larger and more complete datasets, enabling more accurate computational models, more accurate predictions, and improved selections.
To address consequences of climate change and population growth, plant and animal breeding needs to become more efficient and data driven to ensure a healthy, resilient, and sustainable agricultural production system. Modern breeding techniques require large amounts of high quality data to be effective, requiring digital methods for data collection, management, and analysis. They rely on several research disciplines, including plant phenomics, genetics, genomics, and agronomy, involving research institutes, genebanks, and breeding companies. Interoperability between research software tools, systems, and databases can substantially increase the efficiency of a breeding program. The ability to efficiently share data means access to larger and more complete datasets, enabling more accurate computational models, more accurate predictions, and improved selections.

The Breeding API (BrAPI) project is an effort to enable interoperability among breeding tools, systems, and databases. BrAPI is a standardized Representational State Transfer (REST), web service, Application Programming Interface (API), specification for breeding and related agricultural data. [@doi:10.1093/bioinformatics/btz190] By using the BrAPI standard, breeding software can more easily become interoperable, allowing groups to more easily share data and software tools.
The Breeding API (BrAPI) project is an effort to enable interoperability among breeding tools, systems, and databases. BrAPI is a standardized Representational State Transfer (REST), web service, Application Programming Interface (API), specification for breeding, genetics, phenomics and related agricultural data. [@doi:10.1093/bioinformatics/btz190] By using the BrAPI standard, breeding software can more easily become interoperable, allowing groups to more easily share data and software tools.

Since its first publication [@doi:10.1093/bioinformatics/btz190], BrAPI has seen a significant increase in community services, compatible tools, and participating organizations. The community has organized numerous hackathons to evolve the specifications, resulting in continuous improvements and enhancements. This report includes a short technical description of the standard and a showcase of the applications, services, and tools available in the BrAPI community. BrAPI has become an essential part of the digital infrastructure for breeding applications and related agricultural projects. It is the intention of this manuscript to demonstrate the value of BrAPI to the wider scientific community as an effective and efficient means to collaborate and share resources.
Since its first publication [@doi:10.1093/bioinformatics/btz190], BrAPI has seen a significant increase in community services, compatible tools, and participating organizations. The community has organized numerous hackathons to evolve the specifications, resulting in continuous improvements and enhancements. This report includes a short technical description of the standard and a showcase of the applications, services, and tools available in the BrAPI community. BrAPI has become an essential part of the digital infrastructure for breeding, genetics and phenomics applications and related agricultural projects. It is the intention of this manuscript to demonstrate the value of BrAPI to the wider scientific community as an effective and efficient means to collaborate and share resources.

### How it works

An API is a technical connection between two pieces of software. Just as a Graphical User Interface (GUI) or a Command Line Interface (CLI) allows a human user to interact with a piece of software, an API allows one software application to interact with another. A GUI or CLI might allow a user to input data, read data, and start processes within an application. An API allows one piece of software (sometimes called a client, user agent, or service consumer) to programmatically input data, read data, and start process within another piece of software (sometimes called a server or service provider).

A REST style web service is a type of API commonly used in today's modern web infrastructure. REST is a technical architecture that describes the stateless transmission of data between applications. Typically, RESTful web service APIs are implemented using the standard HTTP protocol that most of the modern internet is built upon. These implementations generally use JavaScript Object Notation (JSON) to represent the data being transferred. Both HTTP and JSON are programming language agnostic, very stable, and very flexible. This means BrAPI can be implemented in almost any piece of software, and can solve a wide range of use cases.
A REST style web service is a type of API commonly used in today's web infrastructure. REST is a technical architecture that describes the stateless transmission of data between applications. Typically, RESTful web service APIs are implemented using the standard HTTP protocol that most of the modern internet is built upon. These implementations generally use JavaScript Object Notation (JSON) to represent the data being transferred. Both HTTP and JSON are programming language agnostic, very stable, and very flexible. This means BrAPI can be implemented in almost any piece of software, and can solve a wide range of use cases.

Data repositories and service providers can choose to represent their data as a BrAPI compatible API. By mapping the internal data structures to the standard models, data repositories can easily expose data to the outside world. Similarly, they can accept new data from external sources and automatically map the new data into an existing database. Client application developers can take advantage of this standardization by building tools and connectors that integrate with all BrAPI compatible data repositories. Visualization, reporting, analytics, data collection, and quality control tools can be built once and shared with other organizations following the standards. As the number of BrAPI compatible databases, tools, and organizations grows, so does the value added by implementing the standard into a given application.
Data repositories and service providers can choose to represent their data as a BrAPI compatible API. By mapping their internal data structures to the standard models, data repositories can easily expose data to the outside world. Similarly, they can accept new data from external sources and automatically map the new data into an existing database. Client application developers can take advantage of this standardization by building tools and connectors that integrate with all BrAPI compatible data repositories. Visualization, reporting, analytics, data collection, and quality control tools can be built once and shared with other organizations following the standards. As the number of BrAPI compatible databases, tools, and organizations grows, so does the value added by implementing the standard into a given application.

### Project Updates

Over its lifetime, the BrAPI project has grown and changed substantially. The latest stable version of the specification (v2.1) looks vastly different from the first version (v1.0) released in 2017. The total size of the specification has almost quadrupled in that time, going from 51 endpoints documented in v1.0 to 201 endpoints documented in v2.1. Because of this growth, the specification documents were reorganized into four modules: BrAPI-Core, BrAPI-Germplasm, BrAPI-Genotyping, and BrAPI-Phenotyping. Figure {@fig:domains} shows a simplified domain map of the whole BrAPI v2.1 data model, divided into the organizational modules. The early versions of the specification focused on read-only phenotype data, with a small consideration to the other domains. Now the specification has a full representation of most of the major concepts applicable to the breeding process. The new specification is also internally consistent, easier to navigate, and allows for read, write, and update capabilities. None of those qualities were a guarantee for the earlier versions.
Over its lifetime, the BrAPI project has grown and changed substantially. The latest stable version of the specification (v2.1) looks vastly different from the first version (v1.0) released in 2017. The total size of the specification has almost quadrupled in that time, going from 51 endpoints documented in v1.0 to 201 endpoints documented in v2.1. Because of this growth, the specification documents were reorganized into four modules: BrAPI-Core, BrAPI-Germplasm, BrAPI-Genotyping, and BrAPI-Phenotyping. Figure {@fig:domains} shows a simplified domain map of the whole BrAPI v2.1 data model, divided into the organizational modules. The early versions of the specification focused on read-only phenotype data, with a small consideration to the other domains. Now the specification has a full representation of most of the major concepts applicable to the breeding and research process. The new specification is also internally consistent, easier to navigate, and allows for read, write, and update capabilities. None of those qualities were a guarantee for the earlier versions.

![A simplified domain map of the whole BrAPI data model, divided into organizational modules. A more detailed Entity Relationship Diagram (ERD) is available on brapi.org.](images/BrAPI_Domains_v2-1_vertical.png){#fig:domains width="100%"}

As the specification has matured, so have the tools, services, and libraries available to the community to work with the specification. Every version of the specification is now released with a change log to guide developers upgrading from a previous version, an Entity Relationship Diagram (ERD) to describe the whole data model visually, and a JSON Schema data model to be used in some automated development efforts. For groups who are using Java, Java Script, Python, R, or Drupal, there are community maintained libraries available that contain full BrAPI implementations ready to be added to some existing code. The BrAPI Test Server and the BRAVA validation tool are both still available to the community for testing purposes, and they have been maintained to support every version of the specification. Finally, there are three new resource list pages on brapi.org to advertise the BrAPI compatible software available in the community. The BrAPPs list page, servers list page, and compatible software list page showcase many of the BrAPI compatible applications and data resources available in the community.
As the specification has matured, so have the tools, services, and libraries available to the community to work with the specification. Every version of the specification is now released with a change log to guide developers upgrading from a previous version, an Entity Relationship Diagram (ERD) to describe the whole data model visually, and a JSON Schema data model to be used in some automated development and validation efforts. For groups who are using Java, Java Script, Python, R, or Drupal, there are community maintained libraries available that contain full BrAPI implementations ready to be added to some existing code. The BrAPI Test Server and the BRAVA validation tool are both still available to the community for testing purposes, and they have been maintained to support every version of the specification. Finally, there are three new resource list pages on brapi.org to advertise the BrAPI compatible software available in the community. The BrAPPs list page, servers list page, and compatible software list page showcase many of the BrAPI compatible applications and data resources available in the community.

### Community Growth

The international BrAPI Community consists of software developers, biologists, and other scientists working on BrAPI related projects and data sources. This community is what sustains the BrAPI project, builds implementations, maintains development tools, and provides input to enhance the specification. As the project has grown, so has the community. The BrAPI project started in June 2014 with less than ten people coming together to discuss the idea. Over the next ten years, the community has grown to between 200 and 250 members.

The BrAPI Hackathons are a major staple of the BrAPI community. Twice a year, the community gathers to discuss the specification and collaborate on BrAPI related projects. This time is very valuable to the community; for some organizations, the hackathon is the only time during the year when they have time to work on anything related to BrAPI. During the COVID-19 pandemic, virtual hackathons took the place of in-person events. While the virtual hackathons do not provide the same level of face-to-face time that is crucial to collaborative work, they did allow for more attendees to gather and share their opinions. Going forward, the community leadership has decided to have one in-person hackathon and one virtual hackathon each year, to balance the advantages of both.
The BrAPI Hackathons are a major staple of the BrAPI community. Twice a year, the community gathers to discuss the specification and collaborate on BrAPI related projects. This time is very valuable to the community; for some organizations, the hackathon is the only time during the year when they can collaboratively work on anything related to BrAPI. During the COVID-19 pandemic, virtual hackathons took the place of in-person events. While the virtual hackathons do not provide the same level of face-to-face time that is crucial to collaborative work, they did allow for more attendees to gather and share their opinions. Going forward, the community leadership has decided to have one in-person hackathon and one virtual hackathon each year, to balance the advantages of both.
4 changes: 4 additions & 0 deletions content/03.00.HEADER.Success.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,7 @@
Below are a number of short success stories from the BrAPI community. These tools, applications, and infrastructure projects serve as another indicator of community growth and success over the past 5-10 years. These stories clearly illustrate all the different ways the BrAPI Standard can be used productively and in practice. Figure {@fig:apps} contains a summary of the tools described below.

![A summary of all the tools described below and the general areas each tool is designed to handle](images/BrAPI_Paper_Applications_Chart.png){#fig:apps width="100%"}

<!-- NOTES TO UPDATE THE BrAPI_Paper_Applications_Chart.png :
FAIDARE: Core, Phenotyping, Genotyping, Germplasm Visualization .
-->
Loading

0 comments on commit c0e0238

Please sign in to comment.