add individual parser files

[pratikpakhale]

Final Report: GSoC '24

• Student Name: Pratik Pakhale
• Organization: PEcAn
• Mentors: Henri Kajasilta, Olli Niemitalo, Nihar Sanda
• Project: Adopting Data Schema for Field Management Events

Overview

This PR aims to adapt a data schema for this fieldactivity app. (Fieldactivity is an application that allows field operators and researchers to enter field information about management activities through UI to aid bookkeeping of such events.) These events are used for example in fieldobservatory and the schema is convenient as we only need to update the data structure in one place if needed, and this way the data will also always be compatible with the other data sources that we might have!

It's designed to create forms for various field activities, with support for multiple languages, different input types.

Features

• Dynamic UI generation based on JSON schema
• Multi-language support (English, Finnish, Swedish)
• Support for various input types (select, number, string, textarea)
• Nested object and array handling

File Structure in R/parser/

• fct_parser.R: Contains functions for parsing the JSON schema
• fct_ui.R: Contains functions for creating UI elements
• app_ui.R: Defines the main UI structure of the Shiny app
• app_server.R: Defines the server-side logic of the Shiny app

Overall Workflow:

The JSON schema is loaded and parsed using functions from fct_parser.R.
The parsed schema is then used to generate the UI using functions from fct_ui.R.
The generated UI is rendered in the Shiny application.

JSON Schema Parsing (fct_parser.R):

The parsing process begins with the parse_json_schema function. This function takes a complete JSON schema as input and processes it to create a structured representation that can be used for UI generation and data validation.
The main workflow in fct_parser.R is as follows:

1. parse_json_schema: This is the entry point for schema parsing. It calls parse_event to process the entire schema.
2. parse_event: This function handles individual events in the schema. It extracts the event's title and processes its properties and any 'oneOf' sections. For each property, it calls parse_property.
3. parse_property: This function processes individual properties within an event. It handles various property types (string, number, boolean, etc.) and special cases like 'allOf' (typically used for references) and 'oneOf' (for properties with multiple options). For array items and nested object properties, it recursively calls itself.
4. get_multilingual_field: This helper function extracts multilingual values (English, Finnish, and Swedish) for a given field from an object.

The parsing process creates a structured representation of the schema, organizing information about events, their properties, and any nested structures or options.

UI Generation (fct_ui.R):

The UI generation process takes the parsed schema and creates Shiny UI elements based on it. The main workflow in fct_ui.R is as follows:

1. create_ui: This is the main function for generating the UI. It takes the parsed schema, a namespace function for Shiny module compatibility, and a language code. It iterates through each event in the parsed schema and creates UI elements for both common properties and 'oneOf' sections.
2. create_properties_ui: This function generates UI elements for a set of properties. It handles different property types, including simple types (string, number, boolean) and complex types like 'oneOf' properties and arrays of objects. For simple types, it calls create_widget. For 'oneOf' properties, it creates a select input for choosing between options and generates nested property inputs for each option.
3. create_oneof_ui: This function generates UI elements for 'oneOf' sections in the schema. It creates a select input for choosing between options and generates conditional panels for each option's properties.
4. create_widget: This function creates individual input widgets based on the property type and attributes. It handles various input types such as select inputs, numeric inputs, text inputs, and date inputs.
5. get_select_choices: This helper function extracts and formats choices for select input widgets.

The UI generation process creates a dynamic form based on the parsed schema, with appropriate input widgets for each property and handling complex structures like nested objects and 'oneOf' sections.

The current state

The current state of the PR has the individual parser in the R/parser/ directory. However, you'll need to use these files and run in a new golem environment to be able to run it.

To Do

We still need to implement this parser with management-event-schema.json instead of ui_structure.json. The current implementation is based on the ui_structure file and display_names.csv file from which the UI is created and labels are assigned based on code names. But we need to integrate this parser along with the implementation of the ui_structure file as the other UI elements like site selection, event display table and more are not in the JSON Schema. Then use the existing validation logic, language change observers for the parsed UI.

One way to do that is changing how we store the global form variable. Having a new variable which is schema form specific and then changing the server observers to handle both these variables independetly, yet, compatible with the workflow of the existing app. Then having some attributes in the generated UI to identify how to store these events in a compliant data structure.

Challenges

The major challenge I faced while implementing the parser was during handling oneOf properties and conditionally rendering them. As the global oneOf fields are condtional panels where as the internal property oneOf fields are select elements. Also another issue was managing the select input associated with the conditional properties. We have these oneOf properties, and we also have a main property element which has a oneOf field which represents a normal select element. This all was needed to be handled in a recursive manner to finally build the UI

Apart from that, getting comfortable with the JSON Schema, and planning the architecture of the parser (whether to build it directly inside the main app or as an individual thing took some time)
According to my proposal, the data handling and main app integration parts are still left. The parser as a separate project is complete by the final GSoC evalutaion.

Project Reflection

As I look back on my GSoC '24 project for PEcAn, I can see that the core components of my proposal were:

1. Creating a parser for the JSON schema to generate dynamic UI elements
2. Integrating this parser into the existing Fieldactivity app

These two aspects ended up taking the majority of my time during the GSoC period. The parser development was particularly challenging, especially when it came to handling nested structures, 'oneOf' properties, and conditional rendering. I'm pleased that I was able to successfully implement the parser as a separate project, which can now generate UI elements based on the JSON schema.

However, within the allocated time for the project, I didn't have the opportunity to work on some of the additional features I had originally planned. These included:

1. Implementing the parser with the 'management-event-schema.json' instead of 'ui_structure.json'
2. Integrating the parser with the existing Fieldactivity app, including site selection and event display table

Looking back, I still see these additional features as valuable for the project's overall goals. Implementing them would make the Fieldactivity app more flexible and easier to maintain, as it would allow for dynamic form generation based on a standardized schema. This would improve compatibility with other data sources and make it easier to update the data structure in the future.

While I didn't have time to implement these extra features during the GSoC period, I believe the work I've done on the parser lays a solid foundation. This can serve as a starting point for future development, either by myself (I plan to do so) or other contributors to the PEcAn project.

In reflection, I feel that my project achieved its primary goal of creating a functional parser for the JSON schema, which is a significant step towards improving the Fieldactivity app's flexibility and maintainability. The integration and additional features remain important for the project's future development, and I believe they could be valuable contributions for the PEcAn community going forward.

Overall Experience

One of my favorite aspects of participating in Google Summer of Code (GSoC) has been the incredible exposure to real-world software development projects, especially in R. It's been a fantastic way to apply what I've learned in a practical setting and see the impact of my work.

Mentorship and Community Interaction: I absolutely value the opportunity to connect with mentors, the community and my fellow peers in PEcAn family. The guidance and support from my mentors have really helped a lot. I've learned so much about open-source culture, how to communicate and have developed valuable relationships that I believe will continue to benefit me in the future.

Flexibility and Independence: The program's flexibility is another highlight for me. Being able to work independently from anywhere has been empowering. My mentors understanding my commitments and requests was also one of the things I appreciate the most!