[AutocompleteWidget] Some MeSH IRIs cannot be preselected

[vera]

We currently have a bug related to some mesh IRIs not being properly preselected in the AutocompleteWidget (https://github.com/nfdi4health/csh-ui/issues/1177).

To reproduce, for example, set preselected to [{ iri: "http://id.nlm.nih.gov/mesh/D016019" }].

(e.g. here https://ts4nfdi.github.io/terminology-service-suite/comp/latest/?path=/story/react_search-autocompletewidget--with-value)

You will see that the AutocompleteWidget remains empty.

However, https://semanticlookup.zbmed.de/api/terms?iri=http://id.nlm.nih.gov/mesh/D016019 resolves to the term ("Survival Analysis").

And, for example, the other widgets can also display the term + associated info when given this IRI:

[jusa3]

The problem lies in the AutocompleteWidget code. The preselected property can be an IRI or a label. If it's a label, the label is displayed directly (if the allowCustomTerms property is set to true) - this works fine. If it's an IRI, the associated metadata is retrieved from the OLS select API. So the IRI is set as the search term. This is not a problem for the MeSH Bioportal IRIs as they are in the underlying OLS backend. But for the MeSH NLM IRIs, no results are found (because they're not in the underlying OLS backend and are handled via a mapper gateway) - so no metadata is fetched and no preselected value is displayed.

Another issue we face when moving from the OLS3 to the OLS4 backend: The OLS4 search API doesn't support searching for an IRI as a search term anymore. This drawback should be reported to the OLS4 repo.

Both problems could be solved by not searching for the preselected IRI, but retrieving the metadata information directly via the OLS Term API. I'll be testing this in the next few days.

[johannes-darms]

How can we fix this in the short term? This breaks the edit functionally of our portal more or less.

[vera]

The preselected property can be an IRI or a label. If it's a label, the label is displayed directly (if the allowCustomTerms property is set to true) - this works fine.

Would it be possible to display the label as a fallback value if both a IRI and a label are provided and the IRI resolving fails? I think that might be a good short term fix.

Both problems could be solved by not searching for the preselected IRI, but retrieving the metadata information directly via the OLS Term API. I'll be testing this in the next few days.

Sounds good 👍

[jusa3]

This needs to be a high priority as the keyword editing functionality of existing resources on the Studyhub production instance doesn't work!

The small problem turned out to be a big one:

• OLS3 search endpoint works for an IRI as search term
• The OLS4 search endpoint doesn't work for an IRI as a search term (@rombaum OLS4 issue)
• Both endpoints don't work for the MeSH NLM IRI as a search term because it's not in the OLS backend system.

Ideas for a fix:

• Use the MeSH NLM IRIs in the backend system and adapt the mapper - doesn't work because the resource is not available
• Adapt the mapper to change the search term (if it's a MeSH NLM IRI, convert it to the MeSH Bioportal IRI) - doesn't work for OLS4 because it doesn't support an IRI as a search term
• Adapt the widget to recognize NLM IRIs and map them to the Bioportal IRI - is a very short term solution as it only works for OLS3 - doesn't work for OLS4 because it doesn't support an IRI as a search term - requesting MeSH IRIs is a special case and the widget should remain generic
• Use the label (if available) as a fallback value - great idea, should be implemented anyway to make the widget more robust - is a very short term solution as it only works for OLS3 - may require additional user effort
• Use the term/individuals/property endpoint to get the entity information directly - we need the entity type for this endpoint but can't get it from the IRI. We could make all three calls and use the data from the successful one but a) this would cause more traffic and b) some terminologies use the same IRIs for terms and individuals
• Use the OLS4 v2 entity endpoint to get the entity information - this is now official! (https://www.ebi.ac.uk/ols4/help) - no backwards compatibility with OLS3

I would suggest implementing the label fallback solution as a short term OLS3 fix to restore functionality for the NFDI4Health Studyhub and because it can last as a fallback solution. @vera would the effort on your part be okay?

Regarding the OLS4 issue - the best solution would be a quick fix from the OLS team to allow an IRI as a search term again - as it is possible in OLS3. If they fix it, we can easily adapt the Studyhub specific gateway to map the NLM IRI and Bioportal IRI in both directions.

Any other ideas? @rombaum @Pooya-Oladazimi

Since we planned to replace the OLS3 with the OLS4 backend for the Health Studyhub and Terminology Service in the next days, we should discuss this again.

[Pooya-Oladazimi]

@jusa3

I think using IRI as the search term does not make sense and that's the reason ols4 stopped using it (why do you search if you already have an IRI?). Instead, the thing you suggested already i.e. using entity endpoint in ols4 would be the solution. So we use the entity endpoint. If it returns 404, then the term does not exist anyway and we fall back to the label if it exists.

the challenge is that we need to implement two different mechanism for ols4 and ols3 since ols3 does not have any entity endpoint. I do not see any simple solution for this.

[vera]

I would suggest implementing the label fallback solution as a short term OLS3 fix to restore functionality for the NFDI4Health Studyhub and because it can last as a fallback solution. @vera would the effort on your part be okay?

Yes, this would work for us as a short term fix, we already store and pass the labels to the preselected param anyways. 👍

[jusa3]

Thanks for your comment @Pooya-Oladazimi , that makes it clearer.

You're right, searching by IRI doesn't make sense.

If we migrate the terminology service backend for the Studyhub to OLS4, the OLS3 implementation in the AutocompleteWidget works fine as it is, as long as nobody is using our MeSH resource in an OLS3 backend system (don't now any other use case and for future use cases OLS4 will be preferred and the mapping task must be handled by the any gateway). So we would just need to add the OLS4 entity endpoint integration and add the useLegacy property.

[rombaum]

I think using IRI as the search term does not make sense

I would disagree in this point. We directly see in this issue (thankfully open by @vera) why this make sense. Or in other words since OLS3 didn't supported an endpoint for entities it was possible to get the requested information via the search or the select endpoint. In some case you will have only the IRI and you want to know what this is about. It also make sense in my opinion since you could get all the other information via the terminology service and it would be fine to only save the IRI in you database. So, I would say a terminology service should try to resolve an IRI. And in the case of OLS3 the search or the select endpoints are ideal for this since they are not specific to search a term. For these endpoints it is possible to define a type (in OLS 3 that are property, class, term, individual, and ontology) via an parameter in a request. Since, they want to provide back comparability they should provide the functionality to use the IRI as input. I will also create an issue at the ols repository for this. (@jusa3 thanks for the reminder)

Nevertheless to solve this situation I would suggest that we should use in the case of setting the flag useLegacy to false the entity endpoint. The widget could also check if the input is an IRI and if this is the case we could use the parameter iri instead of search. This resolve the issue and would look like this: https://ols4-nfdi4health.prod.km.k8s.zbmed.de/ols4/api/v2/entities?iri=http://purl.bioontology.org/ontology/MESH/D016019 . And yes for the studyhub this still must be handled via a mapper gateway of SemLookP.

[Pooya-Oladazimi]

@rombaum

the way I see it is one of these two scenarios:

• Target backend runs OLS4: in this case, there is an entity endpoint in v2 that one can use to fetch a term with just an IRI without knowing the type: https://www.ebi.ac.uk/ols4/api/v2/ontologies/efo/entities/http%253A%252F%252Fwww.ebi.ac.uk%252Fefo%252FEFO_1000967?lang=en
• OR, the backend is running ols3 which we can use the search as we are doing right now.

If we want to argue that ols4 v1 has to be backward compatible and support iri search, yes then an issue needs to be created.

[rombaum]

• Target backend runs OLS4: in this case, there is an entity endpoint in v2 that one can use to fetch a term with just an IRI without knowing the type: https://www.ebi.ac.uk/ols4/api/v2/ontologies/efo/entities/http%253A%252F%252Fwww.ebi.ac.uk%252Fefo%252FEFO_1000967?lang=en

@Pooya-Oladazimi you're right with the two scenarios but you didn't need the ontology id (like efo) for the ols4 v2 API since it would work in this direction: https://www.ebi.ac.uk/ols4/api/v2/entities?iri=http://www.ebi.ac.uk/efo/EFO_0000400

[rombaum]

I will also create an issue at the ols repository for this. (@jusa3 thanks for the reminder)

done: EBISPOT/ols4#859