Skip to content

Commit

Permalink
Update docs/future-work/motor-system-improvements/model-based-explora…
Browse files Browse the repository at this point in the history
…tion-policy.md

Co-authored-by: vclay <[email protected]>
  • Loading branch information
nielsleadholm and vkakerbeck authored Jan 21, 2025
1 parent d8a059f commit db25695
Showing 1 changed file with 1 addition and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,6 @@ During exploration/learning-focused movement, we do not make use of any model-ba
- Locations that are useful for distinguishing objects, such as the fork vs. spoon heads, are worth knowing in detail, because they define the differences between similar objects. These points correspond to those that are frequently tested by the hypothesis-testing policy (see [Reuse Hypothesis-Testing Policy Target Points](../motor-system-improvements/reuse-hypothesis-testing-policy-target-points.md)), and such stored locations can be leveraged to guide exploration.
- As we introduce hierarchy, it may be possible to unify these concepts under a single policy, i.e. where frequently changing features can either be at the sensory (low-level) input, or at the more abstract level of incoming sub-objects.

Model-based exploration policies will be particularly important in an unsupervised setting, due to the lack of supervisory support. In particular, Monty will need to efficiently explore objects in order to recognize the commonality between different instances in which they are observed. For example, after learning a coffee mug from one view, and then seeing it from a different angle, Monty can move around to see that these are two views of the same object, and combine these into a single model. Learning the full structure of an object through such active exploration should be easier than relying on separate, idealized views of the object (e.g. 14 rotations corresponding to the 6 faces and 8 corners of a cube), where the relationship between instances may be ambiguous.
Model-based exploration policies will be particularly important in an unsupervised setting. In particular, Monty will need to efficiently explore objects in order to recognize the commonality between different perspectives from which they are observed. For example, after learning a coffee mug from one view, and then seeing it from a different angle, Monty can move around to see that these are two views of the same object, and combine these into a single model. However, this recognition requires some overlap between the parts of the mug that have been see during the first exposure and what is observed on the second exposure. Learning the full structure of an object through active exploration should be easier than relying on separate, idealized views of the object (e.g. 14 rotations corresponding to the 6 faces and 8 corners of a cube), where the overlap between view points may be minimal.

Such active exploration will also be important when dealing with compositional or multi-object settings, where establishing views of all sides of an object is not straightforward due to the presence of other objects or object-parts.

0 comments on commit db25695

Please sign in to comment.