Plausible Unit Concept IDs Revamp

[katy-sadowski]

@clairblacketer @dimshitc I'm starting this thread to keep track of the work to revamp the lists of plausible unit concept IDs in DQD. Feel free to add comments with additional info / corrections to what I've put below.

To summarize what we've discussed, there are 2 types of issues that could impact the plausible unit concept IDs in our threshold files:

1. Vocabulary updates render units on the lists invalid/non-standard, or add new standard concepts that should be on the lists
2. The lists are incomplete or contain inaccurate entries, due to an issue with the mapping process used to generate the lists

As of v2.4.0, we've addressed all known vocabulary-related issues. We will continue to review vocabulary changes impacting standard unit concepts on an annual basis and update the lists as needed. We anticipate such changes to be rare. I've put a reminder on my calendar to do this next year 😃

Work is underway to re-do the mapping of measurements to their associated standard unit concepts. Currently @dimshitc is scoping out different ways to do this, and is speaking with the THEMIS group, who are also interested in standards for measurement<>unit pairs (Forums thread: https://forums.ohdsi.org/t/next-vocabulary-wg-meeting-july-25-ohdsi-measurement-hierarchies/19405).

Related issues:

• Missing plausible unit concept IDs for measurement concepts 3006627, 3015681 (and maybe more) #471
• Measurement - if a record has a value_as_number but no unit_concept_id this would be a failure #288
• measurement/unit check wrong percentage of failed records  #415