Add owlready2-based OntologyConcept #141

ghost · 2024-04-04T19:21:18Z

This PR adds API that enables designators to link themselves with concepts defined in ontologies, specifically ones of OWL format, utilizing owlready2.

-> This will enable robot motion plan to receive abstract ontology concepts as inputs, which will resolve themselves to specific physical objects in world, upon certain conditions/constraints from user scenario requirements.

Main updates are:

Add ontology.py with `OntologyManager, providing various queries into ontology classes and properties, as well as relations among them.
Add ontology_common.py with OntologyConceptHolder wrapping ontology_concept as an owlready2.Thing to avoid inheritance with added attributes that cannot be saved into an OWL file and reused later upon that file reloading.

Notably,

Ontology triple classes can be also created dynamically with predicate relations set up from facts discovered in experimentation/robot training.
Those classes can also optionally inherit from ones existing in loaded ontologies.

DesignatorDescription, ObjectDesignatorDescription, ~ActionDesignatorDescription: add onto_concept_holders A list of holders ontology concepts that the designator type is semantically categorized as or associated with
Add ontology.ipynb, tutorial featuring basic usages of OntologyManager with examples used in robot motion plan
bullet_world_testcase.py, test_action_designator.py: Add tests for ontology concepts loaded from SOMA
requirements.txt add owlready2

@AbdelrhmanBassiouny @tomsch420

…logy concepts from SOMA DesignatorDescription add onto_concept attr as an owlready2 Thing ~Action auto fetch onto_concept from OntologyOWL

TestActionDesignatorGrounding add is-valid-onto_concept test

…tion: onto_concepts

requirements.txt

tomsch420 · 2024-04-05T07:04:42Z

src/pycram/designator.py

@@ -361,6 +364,11 @@ def get_slots(self) -> List[str]:
    def copy(self) -> Type[DesignatorDescription]:
        return self

+    def get_default_onto_concept(self):


return type annotation

src/pycram/designator.py

src/pycram/helper.py

tomsch420 · 2024-04-05T07:14:21Z

src/pycram/ontology.py

+    """
+
+    """
+    The ontology instance as the result of an ontology loading operation


i am fairly confident that docstrings for class variables should be below the variable declaration and not above. Please check that this is rendered as you intended by building the doc.

tomsch420 · 2024-04-05T07:19:52Z

src/pycram/ontology.py

+                                           owlready2.Property]] = owlready2.ObjectProperty):
+        """
+        Dynamically create ontology triple classes under same namespace with the main ontology,
+        aka subject, predicate, object, with relation among them


no abbreviations in docstrings also

tomsch420 · 2024-04-05T07:21:09Z

src/pycram/ontology.py

+        return obj_designator
+
+    @classmethod
+    def demo_placeable_on_candidates_query(cls):


demos should be either testcases or jupyter notebooks in the documentation examples. Please remove all demos from production code.

tomsch420 · 2024-04-05T07:21:37Z

src/pycram/ontology.py

+            print(edible_onto_concept, [des.types for des in edible_onto_concept.designators])
+
+
+if __name__ == "__main__":


pycram internal modules should not have entry points like this.

src/pycram/ontology.py

examples/ontology.ipynb

Tigul · 2024-04-05T07:31:54Z

src/pycram/designators/object_designator.py

@@ -124,6 +125,7 @@ def __init__(self, names: List[str], types: List[str],
        :param reference_frames: Frame of reference in which the object position should be
        :param timestamps: Timestamps for which positions should be returned
        :param resolver: An alternative resolver that resolves the input parameter to an object designator.
+        :param onto_concepts: A list of ontology concepts that the object is categorized as
        """
        super(LocatedObject, self).__init__(names, types, resolver)


There are no onto_concepts here

Tigul · 2024-04-05T07:33:05Z

src/pycram/helper.py

@@ -22,6 +23,13 @@
 import os
 import math

+class Singleton(type):
+    _instances = {}


Also here docstring

the instances dict still needs and explanation

Make owlready2 optional dependency Add test for OntologyManager

used to link an ontology concept, either created dynamically or loaded from ontology, with designators & custom resolving callable

warn if owlready2 is not installed

tomsch420 · 2024-04-10T08:13:55Z

examples/ontology.ipynb

Now the notebook prints thousands of info lines. Make sure that this is what you want to show in the end by building the doc.

Ah right, so I trimmed it down to showing several examples only

tomsch420 · 2024-04-10T08:15:42Z

src/pycram/helper.py

@@ -22,6 +23,13 @@
 import os
 import math

+class Singleton(type):
+    _instances = {}


the instances dict still needs and explanation

src/pycram/ontology/ontology_common.py

src/pycram/designators/action_designator.py

… OntologyConceptHolder

… robust

…e None

Tigul · 2024-05-24T09:00:38Z

requirements-ontology.txt

@@ -0,0 +1,2 @@
+-r requirements.txt


I don't think it is necessary to add another requirements file just for one requirement, just put this in the default requirements.txt

Since some may integrate another reasoning-based methods (eg: neural-network based), I'd say ontology should be an optional plugin to pycram core operations.
I actually added it in requirements.txt but then decided to move out as @tomsch420's suggestion.

Tigul · 2024-05-24T09:01:20Z

.github/workflows/new-pycram-ci.yml

@@ -61,6 +61,7 @@ jobs:
          cd /opt/ros/overlay_ws/src/pycram
          pip3 install -r requirements.txt
          pip3 install -r requirements-resolver.txt
+          pip3 install -r requirements-ontology.txt


Remove this and add the requirement to requirements.txt

Subject to our consensus on whether to make Ontology integration as optional or not.

Tigul · 2024-05-24T09:04:50Z

src/pycram/designator.py

+try:
+    import owlready2
+except ImportError:
+    owlready2 = None


Also add some logging here. This case should not happen if owlready2 is in requirements.txt but just to be sure.

Tigul · 2024-05-24T09:06:31Z

src/pycram/designator.py

        """
        Create a Designator description.

-        :param resolver: The grounding method used for the description. The grounding method creates a location instance that matches the description.
+        :param resolver: The grounding method used for the description. The grounding method creates a location instance


Doc strings of Parameter have to be in one line, otherwise the autoAPI will not render them correctly.

Tigul · 2024-05-24T09:08:45Z

src/pycram/designator.py

@@ -437,14 +449,36 @@ def insert(self, session: Session, *args, **kwargs) -> ORMAction:

            return action

-    def __init__(self, resolver=None):
-        super().__init__(resolver)
+    def __init__(self, resolver=None, ontology_concept_holders: Optional[List[owlready2.Thing]] = None):


The typing for ontology_concept_holders is different from the parent class, I guess this is still valid due to inheritance right?

Tigul · 2024-05-24T10:54:21Z

src/pycram/ontology/ontology_common.py

+    def get_designators_of_ontology_concept(self, ontology_concept_name: str) -> List[DesignatorDescription]:
+        """
+        Get the corresponding designators associated with the given ontology concept
+        :param ontology_concept_name: An ontology concept name


All methods are missing doc strings for the return value of the function

Tigul · 2024-05-24T10:57:18Z

src/pycram/ontology/ontology.py

+from pycram.helper import Singleton
+from pycram.designator import DesignatorDescription, ObjectDesignatorDescription
+
+from pycram.ontology.ontology_common import OntologyConceptHolderStore, OntologyConceptHolder


Make Pycram imports relative

Tigul · 2024-05-24T11:03:20Z

src/pycram/ontology/ontology.py

+        rospy.loginfo(f"Direct Instances: {list(ontology_class.direct_instances())}")
+        rospy.loginfo(f"Inverse Restrictions: {list(ontology_class.inverse_restrictions())}")
+
+    def load_ontology(self, ontology_iri) -> tuple[owlready2.Ontology, owlready2.Namespace]:


Missing type for ontology_iri

Tigul · 2024-05-24T11:03:42Z

src/pycram/ontology/ontology.py

+        return out_classes
+
+    @staticmethod
+    def get_ontology_class_by_ontology(ontology: owlready2.Ontology, class_name: str) -> Type[owlready2.Thing] | None:


Typing with "|"

Tigul · 2024-05-24T11:11:28Z

src/pycram/ontology/ontology.py

+        """
+        Create a designator that belongs to a given ontology concept class
+
+        :param designator_name: Designator name


Designator name might be confusing since it implies that it is the name of the designator itself and not what the designator describe. object_name might be better suited.

Optional as only required if creating Object Designator

… floor

Added geometry2

Add owlready2-based OntologyConcept

duc than added 5 commits April 4, 2024 21:11

add pycharm tips

baae59a

add OntologyOwl class using OwlReady2 to fetch object classes by onto…

e929478

…logy concepts from SOMA DesignatorDescription add onto_concept attr as an owlready2 Thing ~Action auto fetch onto_concept from OntologyOWL

BulletWorldTestCase add OntologyOWL init

399f500

TestActionDesignatorGrounding add is-valid-onto_concept test

OntologyHandheldObject, OntologyPlaceHolderObject + DesignatorDescrip…

ab4181f

…tion: onto_concepts

add ontology.ipynb

7e8bca8

tomsch420 requested changes Apr 5, 2024

View reviewed changes

Tigul reviewed Apr 5, 2024

View reviewed changes

duc than added 3 commits April 5, 2024 22:41

abbrev -> full-named vars

176640b

Make owlready2 optional dependency Add test for OntologyManager

new-pycram-ci.yml install ontology deps

c256249

doc/source/examples.rst Add Ontology notebook section

5db24bb

ghost closed this Apr 5, 2024

ghost deleted the onto_concept branch April 5, 2024 20:53

ghost reopened this Apr 5, 2024

duc than added 3 commits April 5, 2024 22:58

Merge branch 'dev' into onto_concept

830a8a7

update after merge from dev

6c46980

add OntologyConceptHolder, wrapping owlready2.Thing

2a21fc5

used to link an ontology concept, either created dynamically or loaded from ontology, with designators & custom resolving callable

ghost closed this Apr 8, 2024

ghost reopened this Apr 8, 2024

duc than added 4 commits April 8, 2024 22:57

OntologyConceptHolder fix get_ontology_concept_holders_by_class()

f85d0ba

OntologyManager put ontology loading into load_ontology()

47acbf6

Merge branch 'dev' into tmp_onto_concept

e9ac814

OntologyConceptHolder typehint

1e4ab04

warn if owlready2 is not installed

ghost requested review from Tigul and tomsch420 April 9, 2024 19:21

Merge branch 'dev' into tmp_onto_concept

a6971b3

tomsch420 reviewed Apr 10, 2024

View reviewed changes

duc than added 3 commits April 10, 2024 11:00

Ontology.ipynb print just a few examples of classes-query results only

a0bfc51

Add OntologyConceptHolderStore as singleton to store all instances of…

9b526ea

… OntologyConceptHolder

OntologyConceptHolderStore: get_ontology_concept_of_designator() more…

a028b6f

… robust

ghost requested a review from tomsch420 April 10, 2024 09:40

duc than added 7 commits April 10, 2024 11:47

ontology.ipynb correct text

7a3ed97

ontology.ipynb Example2: resolve by ontology_concept_holder to be exact

f1853df

More typehint for None possible return

d6714d4

ontology.ipynb typo

fdba8a4

ontology.ipynb correct text

308bf59

OntologyManager fetch soma, dul safely

b57a166

load_ontology() always return ontology + namespace, which can never b…

4cb9888

…e None

Tigul requested changes May 24, 2024

View reviewed changes

duc than added 3 commits May 27, 2024 16:19

address comment docstrings

f3749ff

Merge branch 'dev' into onto_concept

8f836cb

create_ontology_linked_designator() designator_name -> object_name

ed817eb

Optional as only required if creating Object Designator

ghost requested a review from Tigul May 27, 2024 16:19

duc than added 2 commits May 28, 2024 17:47

Merge branch 'dev' into onto_concept

aef78a2

fix after merge with devel: rem spawning plane object due to existing…

ff5d61f

… floor

Tigul merged commit c25295d into cram2:dev May 31, 2024
2 checks passed

Tadinu mentioned this pull request Jun 10, 2024

OntologyManager: add reason() + test #166

Merged

Tadinu mentioned this pull request Aug 20, 2024

Catch remote ontology loading error by owlready2 #188

Merged

Tadinu mentioned this pull request Sep 5, 2024

[OntologyManager] Use in-memory SQL backend by default #198

Merged

hawkina pushed a commit to hawkina/pycram that referenced this pull request Oct 23, 2024

Merge pull request cram2#141 from pkehr/binder-all-robots-geometry

91dd3bb

Added geometry2

mkhoshnam pushed a commit to mkhoshnam/pycram that referenced this pull request Nov 26, 2024

Merge pull request cram2#141 from duc89/onto_concept

0640d5a

Add owlready2-based OntologyConcept

mkhoshnam pushed a commit to mkhoshnam/pycram that referenced this pull request Nov 26, 2024

Merge pull request cram2#141 from duc89/onto_concept

832b3b1

Add owlready2-based OntologyConcept

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add owlready2-based OntologyConcept #141

Add owlready2-based OntologyConcept #141

ghost commented Apr 4, 2024 •

edited by ghost

Loading

tomsch420 Apr 5, 2024

tomsch420 Apr 5, 2024

tomsch420 Apr 5, 2024

tomsch420 Apr 5, 2024

tomsch420 Apr 5, 2024

Tigul Apr 5, 2024

Tigul Apr 5, 2024

tomsch420 Apr 10, 2024

tomsch420 Apr 10, 2024

ghost Apr 10, 2024

tomsch420 Apr 10, 2024

Tigul May 24, 2024

ghost May 24, 2024

Tigul May 24, 2024

ghost May 24, 2024

Tigul May 24, 2024

Tigul May 24, 2024

Tigul May 24, 2024

Tigul May 24, 2024

Tigul May 24, 2024

Tigul May 24, 2024

Tigul May 24, 2024

Tigul May 24, 2024

		print(edible_onto_concept, [des.types for des in edible_onto_concept.designators])


		if __name__ == "__main__":

Add owlready2-based OntologyConcept #141

Add owlready2-based OntologyConcept #141

Conversation

ghost commented Apr 4, 2024 • edited by ghost Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ghost commented Apr 4, 2024 •

edited by ghost

Loading