Valentin Guignon
@@ -478,7 +508,7 @@
Introduction
* Standardization of endpoints across objects - said in a non-tech way
-->
-Plant and animal breeding is an incredibly important part of today’s society. Almost every country in the world has some kind of breeding program supporting the agricultural community to produce bigger, better, healthier, more sustainable crops. Modern breeding techniques require large amounts of high quality data to be effective. In the digital age, that breeding data is being collected, managed, and analyzed with computer software. Interoperability between breeding software tools, systems, and databases can substantially increase the efficiency of a breeding program. The ability to share tools gives each program a boost in computational power. The ability to share data means everyone has access to larger, more complete, datasets and get build more accurate computational models and produce more accurate predictions.
+Plant and animal breeding is an incredibly important part of today’s society. Almost every country in the world has some kind of breeding program supporting the agricultural community to produce bigger, better, healthier, and more sustainable crops. Modern breeding techniques require large amounts of high quality data to be effective. In the digital age, that breeding data is being collected, managed, and analyzed with computer software. Interoperability between breeding software tools, systems, and databases can substantially increase the efficiency of a breeding program. The ability to share tools gives each program a boost in computational power. The ability to share data allows the community to leverage access to larger, more complete, datasets; similarly, the community can build more accurate computational models and produce more accurate predictions.
The Breeding API (BrAPI) project is an effort to enable interoperability among breeding tools, systems, and databases. BrAPI is a standardized Representational State Transfer (REST), web service, Application Programming Interface (API), specification for breeding and related agricultural data. [1] By using the BrAPI standard, breeding software can more easily become interoperable, allowing groups to more easily share data and software tools.
How it works
An Application Programming Interface (API) is a technical connection between two pieces of software. Just as a Graphical User Interface (GUI) or a Command Line Interface (CLI) allows a human user to interact with a piece of software, an API allows one software application to interact with another. A GUI or CLI might allow a user to input data, read data, and start processes within an application. An API allows one piece of software (sometimes called a client, user agent, or service consumer) to programmatically input data, read data, and start process within another piece of software (sometimes called a server or service provider).
@@ -494,7 +524,7 @@ Project Updates
As the specification has matured, so have the tools, services, and libraries available to the community to work with the specification. Every version of the specification is now released with a change log to guide developers upgrading from a previous version, an Entity Relationship Diagram (ERD) to describe the whole data model visually, and a JSON Schema version of the model to be used in some automated development efforts. For groups who are using Java, Java Script, Python, R, or Drupal, there are community maintained libraries available that contain full BrAPI implementations ready to be added to some existing code. The BrAPI Test Server and the BRAVA validation tool are both still available to the community for testing purposes, and they have been maintained to support every version of the specification. Finally, the three new resources list pages on brapi.org advertise the other BrAPI compatible software available in the community. The BrAPPs list displays the 10 standalone, plug-and-play, applications available to the community. The servers list displays the 27 registered public data servers, their current status, and a form for registering additional servers. The compatible software list shows the 31 software applications that are BrAPI compliant, and again, a form for registering additional applications. The process for registering new items for these lists is completely voluntary, so the totals represent a lower bound for the number of BrAPPs, data repositories, and applications available in the the community.
-The international BrAPI Community consists of the software developers, breeders, and related scientists working on BrAPI related projects and data sources. This community is what sustains the BrAPI project, building implementations, maintaining development tools, and providing input to enhance the specification. As the project has grown, so has the community. The BrAPI project started in June 2014 with less than ten people coming together to discuss the idea. Over the next nine years, the community has grown to between 200 and 250 members. The community mailing list has 208 members, and the BrAPI Slack workspace has 234 members. The project leadership uses the mailing list to broadcast newsletters, announcements, and updates to the community. The BrAPI Slack workspace allows members of the community to discuss specific topics and collaborate directly with each other.
+The international BrAPI Community consists of software developers, breeders, and related scientists working on BrAPI related projects and data sources. This community is what sustains the BrAPI project, building implementations, maintaining development tools, and providing input to enhance the specification. As the project has grown, so has the community. The BrAPI project started in June 2014 with less than ten people coming together to discuss the idea. Over the next nine years, the community has grown to between 200 and 250 members. The community mailing list has 208 members, and the BrAPI Slack workspace has 234 members. The project leadership uses the mailing list to broadcast newsletters, announcements, and updates to the community. The BrAPI Slack workspace allows members of the community to discuss specific topics and collaborate directly with each other.
The BrAPI Hackathons are a major staple of the BrAPI community. Twice a year, the community gathers to discuss the specification and collaborate on BrAPI related projects. This time is very valuable to the community; for some organizations, the hackathon is the only time during the year when they have time to work on anything related to BrAPI. During the COVID-19 pandemic, virtual hackathons took the place of in person events. While the virtual hackathons do not provide the same level of face-to-face time that is crucial to collaborative work, they did allow for more attendees to gather and share their opinions. The typical virtual hackathon has about twice as many registered attendees compared to an in-person hackathon. However, attendees have reported much more productive work time during the in-person events. As a compromise, going forward, the community leadership has decided to have one in-person hackathon and one virtual hackathon each year, to balance the advantages of both.
As the project has matured, a formal project leadership structure became increasingly important. As described in Figure 2, the project governance is divided into two groups. The Project Management team is responsible for the day to day operations of the project. The PI and Co-PI are responsible for the project funds, and responsible for hiring the BrAPI Project Coordinator who is paid by the project funds. The Advisory Board is a group of elected officials representing the community. The board is responsible for long term planning of the project, as well as quick decision making on behalf of the community. The two groups meet quarterly to report on progress and stay synchronized.
@@ -514,8 +544,8 @@
Data Collection
* future related use cases, areas to improve -->
Field Book
-
Phenotypic data collection is an essential part of the breeding process. Historically, gathering data in the field was done with pen and paper, or perhaps some version of a digital spreadsheet. The abundance and prevalence of smart phones has allowed the Field Book mobile app to enhance data collection. Field Book can create well formed digital observation records from the moment they are taken. This can improve the efficiency of data collection and reduce human error.
-
BrAPI was able to take things a step further by automating the flow of data from the Field Book mobile app to a central database server. In this work flow, data is collected and stored completely digitally with little-to-no human involvement.
+
Phenotypic data collection is an essential part of the breeding process. Historically, gathering data in the field was done with pen and paper, or perhaps some version of a digital spreadsheet. The abundance and prevalence of smart phones has allowed the Field Book mobile app to enhance data collection. Field Book can create well-formed digital observation records from the moment they are taken. This can improve the efficiency of data collection and reduce human error.
+
In 2018, BrAPI was introduced into Field Book; specifically, the Core and Phenotyping modules. BrAPI was able to take things a step further by automating the flow of data from the Field Book mobile app to a central database server. This workflow allows data collection and storage to be expedited, removing the need of the user to transfer export files manually. Since Field Book’s adoption of BrAPI, many community servers have been integrated to simplify data storage. In this work flow, data is collected and stored completely digitally with little-to-no human involvement.
ClimMob
Not all data can be collected by a single person, or even by a single organization. ClimMob is a tool to easily allow citizen scientists to assist in the data collection process. Although this data may not be as detailed as a focused scientific program, it can be very useful to collect simple data from a wide range of locations and environments.
@@ -561,8 +591,9 @@
DeltaBreed
Pedigree Viewer https://github.com/solgenomics/BrAPI-Pedigree-Viewer
BMS
-
-
brapi sync and other things
+
The Breeding Management System (BMS), developed by the Integrated Breeding Platform (IBP), is a suite of tools designed to enhance the efficiency and effectiveness of plant breeding. BMS covers all stages of the breeding process, with the emphasis on germplasm management and ontology-harmonized phenotyping. It also features analytics and decision-support tools. With its focus on interoperability, BMS integrates smoothly with BrAPI, facilitating easy connections with a broad array of complementary tools and databases, notably Gigwa which is deployed together with the BMS to fulfill the genotyping data management needs of BMS users.
+
The brapi-sync tool, a significant component of BMS’s BrAPI capabilities, was developed by the IBP and released as a BrAPP for community use. Brapi-sync is designed to enhance collaboration among partner institutes within a network such as Innovation and Plant Breeding in West Africa (IAVAO), by enabling the sharing of germplasm and trials across BrAPI-enabled systems. This tool helps overcome traditional barriers to collaboration, ensuring data that was once isolated within specific programs or platforms can now be easily shared, integrated, and synchronized.
+
Additionally, brapi-sync improves data management by utilizing the externalReferences field to maintain links to the origin IDs of each entity it transmits. This not only retains the original context of the data but also establishes a traceability mechanism for accurate data source attribution and verification. Such practices are crucial for maintaining data integrity and fostering trust among collaborative partners, ensuring access to accurate, reliable, and current information.
Breedbase
Since the inception of BrAPI in 2014, the Breedbase group has been a heavy contributor to the BrAPI community.
@@ -603,6 +634,9 @@
AGENT Portal
IPK-Genebank
Agrosystem Integration of germplasm collections in context of data trustee models among private economy and public research, integration of ex-situ genebanks (EU H2020 projects AGENT, INCREASING), integrated agrosystems and plant research infrastructure
+
MIAPPE ISA to BrAPI service
+
Phenotyping is crucial in the breeding process as it enables the identification of desirable traits, selection of breeding lines, and evaluation of breeding success. In the plant community, MIAPPE (Minimal Information About a Plant Phenotyping Experiment) [16] is the established standard for phenotyping experiments and is commonly serialized as ISA Tab [17]. Although ISA Tab is easy to read for non-technical experts due to its file-based approach, it lacks programmatic access, particularly for web applications. BrAPI, which is aligned with MIAPPE, can help solve this problem.
+MIRA is a tool that enables the automatic deployment of a BrAPI server on a MIAPPE-compliant dataset in ISA Tab format. It can be deployed from a Docker image with the dataset mounted. By utilizing the mapping between MIAPPE, ISA, and BrAPI, there is no need for parsing or manual mapping of datasets that are already compliant with (meta-)data standards. By gaining programmatic access through BrAPI to these datasets, it facilitates the integration of phenotyping datasets into web applications.
MIAPPE “BrAPI to ISA” service
Since the release of BrAPI 1.3, efforts have been made to incorporate support for the Minimum Information About Plant Phenotyping Experiments (MIAPPE) standard into the specification [16]. This integration was finalized in BrAPI 2.0, resulting in full compatibility between the two standards. Consequently, BrAPI now encompasses all attributes necessary for MIAPPE compliance, adhering to standardized descriptions in accordance with MIAPPE guidelines. Leveraging BrAPI as a standardized RESTful web service API specification, we employ the ISA standard for storing metadata and phenotyping data in a standardized manner. This data is structured in the ISA-TAB file format and subjected to validation using the MIAPPE ISA configuration. The “BrAPI to ISA” service functions as a converter between BrAPI RESTful endpoints and ISA-TAB, facilitating the archiving of metadata and data and thereby enhancing data preservation and accessibility. The BrAPI2ISA tool is designed to be compatible with BrAPI 1.3, and we invite contributions from the community to extend support for the latest versions of BrAPI.
Data visualization
@@ -613,7 +647,7 @@
Data visualization
* future related use cases, areas to improve -->
Flapjack
-
Flapjack [17] is a multi-platform desktop application for data visualization and breeding analysis (eg, pedigree verification, marker-assisted backcrossing and forward breeding) using high-throughput genotype data. Data can be easily imported into Flapjack from any BrAPI compatible data source with genotype data available. Flapjack Bytes is a smaller, lightweight and fully web-based counterpart to Flapjack, which can be easily embedded into a database website to provide similar visualizations online. Traditionally supporting its own text-based data formats, Flapjack’s use of BrAPI has streamlined the end-user experience for data import and work is underway to determine the best methods to exchange analysis results using future versions of the API.
+
Flapjack [18] is a multi-platform desktop application for data visualization and breeding analysis (eg, pedigree verification, marker-assisted backcrossing and forward breeding) using high-throughput genotype data. Data can be easily imported into Flapjack from any BrAPI compatible data source with genotype data available. Flapjack Bytes is a smaller, lightweight and fully web-based counterpart to Flapjack, which can be easily embedded into a database website to provide similar visualizations online. Traditionally supporting its own text-based data formats, Flapjack’s use of BrAPI has streamlined the end-user experience for data import and work is underway to determine the best methods to exchange analysis results using future versions of the API.
Helium
Helium is a platform for reviewing and managing pedigree data. It is BrAPI enabled, so pedigree data can be viewed straight from a database.
@@ -659,8 +693,8 @@
MGIS
MGIS has germplasm and genotype data stored for many musa accessions. Through BrAPI, users are able to access this data directly from MusaBase, for use in specific experiments.
GIGWA
-
Gigwa is a JEE web application providing means to centralize, share, finely filter, and visualize high-throughput genotyping data [18]. Built on top of MongoDB, it is scalable and can support working smoothly with datasets containing billions of genotypes. Installable from docker images or all-in-one bundle archives, it is pretty straightforward to deploy on servers or local computers and has thus been adopted by numerous research institutes from around the world. Notably, Gigwa serves as a collaborative management tool and/or a portal for exposing the data for genebanks and breeding programs for some CGIAR centers [19]. Thus, the amount of data hosted and made widely accessible using this system has kept growing over the last few years.
-
Gigwa developers have been involved in the BrAPI community since 2016 and took part in designing the genotype-related part of the API’s specifications. Its first BrAPI-compliant features were designed for compatibility with the Flapjack visualization tool [17] and thus primarily turned it into a BrAPI datasource. Consequently, over time, Gigwa being the first and most reliable application implementing BrAPI-Genotyping server calls, local collaborators and even external partners used it as a reference solution to design a number of tools taking advantage of those features (e.g., BeegMac, SnpClust, QBMS). But further use-cases also required Gigwa to be able to consume data from other BrAPI servers, which led to also implement API-client features into the system. Thanks to all this work, a close collaboration was progressively established with the Integrated Breeding Platform team developing the widely used Breeding Management System, that ended up in both applications now being frequently deployed together, Gigwa pulling germplasm or sample metadata from BMS, and BMS displaying Gigwa-hosted genotypes within its own UI.
+
Gigwa is a JEE web application providing means to centralize, share, finely filter, and visualize high-throughput genotyping data [19]. Built on top of MongoDB, it is scalable and can support working smoothly with datasets containing billions of genotypes. Installable from docker images or all-in-one bundle archives, it is pretty straightforward to deploy on servers or local computers and has thus been adopted by numerous research institutes from around the world. Notably, Gigwa serves as a collaborative management tool and/or a portal for exposing the data for genebanks and breeding programs for some CGIAR centers [20]. Thus, the amount of data hosted and made widely accessible using this system has kept growing over the last few years.
+
Gigwa developers have been involved in the BrAPI community since 2016 and took part in designing the genotype-related part of the API’s specifications. Its first BrAPI-compliant features were designed for compatibility with the Flapjack visualization tool [18] and thus primarily turned it into a BrAPI datasource. Consequently, over time, Gigwa being the first and most reliable application implementing BrAPI-Genotyping server calls, local collaborators and even external partners used it as a reference solution to design a number of tools taking advantage of those features (e.g., BeegMac, SnpClust, QBMS). But further use-cases also required Gigwa to be able to consume data from other BrAPI servers, which led to also implement API-client features into the system. Thanks to all this work, a close collaboration was progressively established with the Integrated Breeding Platform team developing the widely used Breeding Management System, that ended up in both applications now being frequently deployed together, Gigwa pulling germplasm or sample metadata from BMS, and BMS displaying Gigwa-hosted genotypes within its own UI.
Client BrAPI libraries being available for R, community members typically write ad-hoc scripts syndicating data from multiple BrAPI sources (for instance phenotypes from a datasource and genotypes from another) in order to run various kinds of analyses such as GWAS, genomic selection or phylogenetic investigations. As a perspective, we may expect the most generic and widely-used of those pipelines to be at least publicly distributed, and possibly web-interfaced using solutions like R-Shiny in order to provide new, excitingly useful online services, based on Gigwa-hosted data.
PHG
@@ -766,14 +800,17 @@
References
16.
Enabling reusability of plant phenomic datasets with MIAPPE 1.1 Evangelia A Papoutsoglou, Daniel Faria, Daniel Arend, Elizabeth Arnaud, Ioannis N Athanasiadis, Inês Chaves, Frederik Coppens, Guillaume Cornut, Bruno V Costa, Hanna Ćwiek‐Kupczyńska, … Cyril Pommier
New Phytologist (2020-04-25)
https://doi.org/gjqcmb
+
+
17.
Toward interoperable bioscience data Susanna-Assunta Sansone, Philippe Rocca-Serra, Dawn Field, Eamonn Maguire, Chris Taylor, Oliver Hofmann, Hong Fang, Steffen Neumann, Weida Tong, Linda Amaral-Zettler, … Winston Hide
Nature Genetics (2012-01-27)
https://doi.org/fxp5gk
+
-
17.
Flapjack—graphical genotype visualization Iain Milne, Paul Shaw, Gordon Stephen, Micha Bayer, Linda Cardle, William TB Thomas, Andrew J Flavell, David Marshall
Bioinformatics (2010-10-18)
https://doi.org/cdvnt8
+
18.
Flapjack—graphical genotype visualization Iain Milne, Paul Shaw, Gordon Stephen, Micha Bayer, Linda Cardle, William TB Thomas, Andrew J Flavell, David Marshall
Bioinformatics (2010-10-18)
https://doi.org/cdvnt8
-
18.
Gigwa v2—Extended and improved genotype investigator Guilhem Sempéré, Adrien Pétel, Mathieu Rouard, Julien Frouin, Yann Hueber, Fabien De Bellis, Pierre Larmande
GigaScience (2019-05-01)
https://doi.org/gtp5bz
+
19.
Gigwa v2—Extended and improved genotype investigator Guilhem Sempéré, Adrien Pétel, Mathieu Rouard, Julien Frouin, Yann Hueber, Fabien De Bellis, Pierre Larmande
GigaScience (2019-05-01)
https://doi.org/gtp5bz
-
19.
A digital catalog of high‐density markers for banana germplasm collections Mathieu Rouard, Julie Sardos, Guilhem Sempéré, Catherine Breton, Valentin Guignon, Ines Van den Houwe, Sebastien C Carpentier, Nicolas Roux
PLANTS, PEOPLE, PLANET (2021-03)
https://doi.org/gtp5bx
+
20.
A digital catalog of high‐density markers for banana germplasm collections Mathieu Rouard, Julie Sardos, Guilhem Sempéré, Catherine Breton, Valentin Guignon, Ines Van den Houwe, Sebastien C Carpentier, Nicolas Roux
PLANTS, PEOPLE, PLANET (2021-03)
https://doi.org/gtp5bx
+feserm
+