New form retrieval #15
Conversation
|
Thanks, for the detailed examples, @anjus1313 ! It's good to know about this issue. I tried to test this with your corresponding PR in the front end today, but I was having trouble getting things working in a way that made sense with the multi-page. Let's test it together next week. |
ginic
force-pushed
the
new-form-retrieval
branch
from
cf314e5 to
3bfa03f
Compare
… generate_key_value_pairs and dataElement_list&categoryOptionCombo list retreival from form information
anjus1313
force-pushed
the
new-form-retrieval
branch
from
3bfa03f to
04423c9
Compare
|
I have added all code changes from the correct_field_names-correction branch in MSR-OCR-streamlit repo into this branch. Now the correct field names, generate key value pairs and upload to DHIS2 functionality works with multi-page functionality. No major code changes from the last commit [e4a1122] in correct_field_names-correction branch by @ginic except for variable name changes and one line change to update session states only for tables that have been edited. |
|
I added the changes we discussed in office hours today. So now the way it should work is:
Anju and I talked about removing the correct field names button, but we're leaning towards not doing that because it requires header data to be entered and it seems jarring for the user to see it change. However, with the async form data, maybe we should revisit? |
| categoryOptionCombos = categoryCombos_to_name_to_id[categoryCombo] | ||
| category_id = categoryOptionCombos[category] | ||
|
|
||
| if cell_value is not None and cell_value!="-" and cell_value!="": |
There was a problem hiding this comment.
Yeah, good check. I wonder if we should even remove those during preprocessing, like the same time we evaluate the math expressions. But that's a task for later.
app_llm.py
Outdated
| @@ -5,6 +5,7 @@ | |||
|
|
|||
| import requests | |||
| import streamlit as st | |||
| from pip._vendor.requests.auth import HTTPBasicAuth | |||
There was a problem hiding this comment.
Let's import the requests version of this instead. The single _ import usually means that the module is only for internal use (see https://realpython.com/python-double-underscore/), so it might not be safe to rely on.
| from pip._vendor.requests.auth import HTTPBasicAuth | |
| from requests.auth import HTTPBasicAuth |
|
@anjus1313 , @rdziewietin, @ByteYJ I made several updates to the new authentication code, so I'd appreciate it if someone besides me tested everything out before we merge to main. Everything looks good to me though! |
|
@ginic The code works well for all authentication steps. Thanks for making changes so quickly in many parts of the code. |
Added getFormJson function to retrieve the entire form information when dataset, orgunit and period are given. This information is now used to generate key value pairs as well as get the list of dataElements and categories for doing field name correction using Levenshtein distance.
Images that have a different field name in metadata from the one on tally sheet:
Tally sheet name has been modified to the one in metadata by correct field names function:
Fields in metadata
Pop1: Resident (Population 1 in metadata) gets changed to Pop1: Resident Discontinued user with correct field names
Fields retrived from metadata