diff --git a/docs/future-work/cmp-hierarchy-improvements/add-top-down-connections.md b/docs/future-work/cmp-hierarchy-improvements/add-top-down-connections.md index 0ec7d570..bcd0d2e4 100644 --- a/docs/future-work/cmp-hierarchy-improvements/add-top-down-connections.md +++ b/docs/future-work/cmp-hierarchy-improvements/add-top-down-connections.md @@ -6,4 +6,6 @@ In Monty systems, low-level LMs project to high-level LMs, where this projection For example, a high-level LM of a dinner-set might have learned that the fork is present at a particular location in its internal reference frame. When at that location, it would therefore predict that the low-level LM should be sensing a fork, enabling the perception of a fork in the low-level LM even when there is a degree of noise or other source of uncertainty in the low-level LM's representation. -In the brain, these top-down projections correspond to L6 to L1 connections, where the synapses at L1 would support predictions about object ID. However, these projections also form local synapses en-route through the L6 layer of the lower-level cortical column. In a Monty LM, this would correspond to the top-down connection predicting not just the object that the low-level LM should be sensing, but also the specific location that it should be sensing it at. This could be complemented with predicting a particular pose of the low-level object (see [Use Better Priors for Hypothesis Initialization](../learning-module-improvements/use-better-priors-for-hypothesis-initialization.md)). \ No newline at end of file +In the brain, these top-down projections correspond to L6 to L1 connections, where the synapses at L1 would support predictions about object ID. However, these projections also form local synapses en-route through the L6 layer of the lower-level cortical column. In a Monty LM, this would correspond to the top-down connection predicting not just the object that the low-level LM should be sensing, but also the specific location that it should be sensing it at. This could be complemented with predicting a particular pose of the low-level object (see [Use Better Priors for Hypothesis Initialization](../learning-module-improvements/use-better-priors-for-hypothesis-initialization.md)). + +This location-specific association in both models is key to how we believe compositional objects are represented. For example, if you had a coffee mug with a logo on it, that logo might make an (unusual) 90 degree bend half-way along its length. This could be learned by associating the logo with the mug multiple times, where different locations in the logo space, as well as different poses of the logo, would be associated with the logo depending on the location on the mugs surface. \ No newline at end of file diff --git a/docs/future-work/cmp-hierarchy-improvements/figure-out-performance-measure-and-supervision-in-heterarchy.md b/docs/future-work/cmp-hierarchy-improvements/figure-out-performance-measure-and-supervision-in-heterarchy.md index 559c61b7..3e6fb6cd 100644 --- a/docs/future-work/cmp-hierarchy-improvements/figure-out-performance-measure-and-supervision-in-heterarchy.md +++ b/docs/future-work/cmp-hierarchy-improvements/figure-out-performance-measure-and-supervision-in-heterarchy.md @@ -1,7 +1,7 @@ --- title: Figure out Performance Measures and Supervision in Heterarchy --- -As we introduce hierarchy and compositional objects, such as a dinner-table setting, we need to figure out both how to measure the performance of the system, and how to supervise the learning. For the latter, we might choose to train the system on component objects in isolation (a fork, a knife, etc.) before then showing Monty the full compositional object (the dinner-table setting). When evaluating performance, we might then see how well the system retrieves representations at different levels of the hierarchy. However, in the more core setting of unsupervised learning, representations of the sub-objects would likely also emerge at the high level (a coarse knife representation, etc.), while we may also find some representations of the dinner scene in low-level LMs. Deciding then how we measure performance will be more difficult. +As we introduce hierarchy and compositional objects, such as a mug with a logo on it, we need to figure out both how to measure the performance of the system, and how to supervise the learning. For the latter, we might choose to train the system on component objects in isolation (various logos, mugs, bowls, etc.) before then showing Monty the full compositional object (a mug with a logo on it). When evaluating performance, we might then see how well the system retrieves representations at different levels of the hierarchy. However, in the more core setting of unsupervised learning, representations of the sub-objects would likely also emerge at the high level (a coarse logo representation, etc.), while we may also find some representations of the mug in low-level LMs. Deciding then how we measure performance will be more difficult. When we move to objects with less obvious composition (i.e. where the sub-objects must be disentangled in a fully unsupervised manner), representations will emerge at different levels of the system that may not correspond to any labels present in our datasets. For example, handles, or the head of a spoon, may emerge as object-representations in low-level LMs, even though the dataset only recognizes labels like "mug" and "spoon". diff --git a/docs/future-work/environment-improvements/make-dataset-to-test-compositional-objects.md b/docs/future-work/environment-improvements/make-dataset-to-test-compositional-objects.md index fbb4612f..41268723 100644 --- a/docs/future-work/environment-improvements/make-dataset-to-test-compositional-objects.md +++ b/docs/future-work/environment-improvements/make-dataset-to-test-compositional-objects.md @@ -2,11 +2,13 @@ title: Make Dataset to Test Compositional Objects --- -We have developed an initial dataset based on recognizing a variety of dinner table sets with different arrangements of plates and cutlery. For example, the objects can be arranged in a normal setting, or aligned in a row (i.e. not a typical dinner-table setting). Similarly, the component objects can be those of a modern dining table, or those from a "medieval" time-period. As such, this dataset can be used to test the ability of Monty systems to recognize compositional objects based on the specific arrangement of objects, and to test generalization to novel compositions. +To test compositional objects, we would like to develop a minimal dataset based on common objects (such as mugs and bowls) with logos on their surfaces. This will enable us to learn on the component objects in isolation, while moving towards a more realistic setting where the component objects must be disentangled from one another. The logo-on-surface setup also enables exploring interesting challenges of object distortion, and learning multiple location-specific associations, such as when a logo has a 90 degree bend half-way along its length. -By using explicit objects to compose multi-part objects, this dataset has the advantage that we can learn on the component objects in isolation, using supervised learning signals if necessary. It's worth noting that this is often how learning of complex compositional objects takes place in humans. For example, when learning to read, children begin by learning individual letters, which are themselves composed of a variety of strokes. Only when letters are learned can they learn to combine them into words. More generally, disentangling an object from other objects is difficult without the ability to interact with it, or see it in a sufficient range of contexts that it's separation from other objects becomes clear. +It's worth noting that observing objects and sub-objects in isolation is often how compositional objects are learned in humans. For example, when learning to read, children begin by learning individual letters, which are themselves composed of a variety of strokes. Only when letters are learned can they learn to combine them into words. More generally, disentangling an object from other objects is difficult without the ability to interact with it, or see it in a sufficient range of contexts that its separation from other objects becomes clear. -However, we would eventually expect compositional objects to be learned in an unsupervised manner. When this is consistently possible, we can consider more diverse datasets where the component objects may not be as explicit. At that time, the challenges described in [Figure out Performance Measure and Supervision in Heterarchy](../cmp-hierarchy-improvements/figure-out-performance-measure-and-supervision-in-heterarchy.md) will become more relevant. +We would eventually expect compositional objects to be learned in an unsupervised manner, such as that a wing on a bird is a sub-object, even though it may never have been observed in isolation. When this is consistently possible, we can consider more diverse datasets where the component objects may not be as explicit. At that time, the challenges described in [Figure out Performance Measure and Supervision in Heterarchy](../cmp-hierarchy-improvements/figure-out-performance-measure-and-supervision-in-heterarchy.md) will become more relevant. + +In the future, we will move towards policies that change the state of the world. At this time, an additional dataset that may prove useful is a "dinner-table setting" with different arrangements of plates and cutlery. For example, the objects can be arranged in a normal setting, or aligned in a row (i.e. not a typical dinner-table setting). Similarly, the component objects can be those of a modern dining table, or those from a "medieval" time-period. As such, this dataset can be used to test the ability of Monty systems to recognize compositional objects based on the specific arrangement of objects, and to test generalization to novel compositions. Because of the nature of the objects, they can also be re-arranged in a variety of ways, which will enable testing policies that change the state of the world. ![Dinner table set](../../figures/future-work/dinner_variations_standard.png) *Example of compositional objects made up of modern cutlery and plates.* diff --git a/docs/future-work/learning-module-improvements.md b/docs/future-work/learning-module-improvements.md index e50dbc83..c6eab29b 100644 --- a/docs/future-work/learning-module-improvements.md +++ b/docs/future-work/learning-module-improvements.md @@ -7,7 +7,7 @@ We have a guide on [customizing learning modules](learning-module-improvements/c These are the things we would like to implement: - [Use off-object observations](learning-module-improvements/use-off-object-observations.md) #numsteps #multiobj -- [Reinitialize hypotheses when starting to recognize a new object](learning-module-improvements/reinitialize-hypotheses-when-starting-to-recognize-a-new-object.md) #multiobj +- [Implement and test rapid evidence decay as form of unsupervised memory resetting](learning-module-improvements/implement-and-test-rapid-evidence-decay-as-form-of-unsupervised-memory-resetting.md) #multiobj - [Improve bounded evidence performance](learning-module-improvements/improve-bounded-evidence-performance.md) #multiobj - [Use models with fewer points](learning-module-improvements/use-models-with-fewer-points.md) #speed #generalization - [Make it possible to store multiple feature maps on one graph](learning-module-improvements/make-it-possible-to-store-multiple-feature-maps-on-one-graph.md) #featsandmorph diff --git a/docs/future-work/learning-module-improvements/implement-and-test-rapid-evidence-decay-as-form-of-unsupervised-memory-resetting.md b/docs/future-work/learning-module-improvements/implement-and-test-rapid-evidence-decay-as-form-of-unsupervised-memory-resetting.md new file mode 100644 index 00000000..eb48fc8b --- /dev/null +++ b/docs/future-work/learning-module-improvements/implement-and-test-rapid-evidence-decay-as-form-of-unsupervised-memory-resetting.md @@ -0,0 +1,13 @@ +--- +title: Implement and Test Rapid Evidence Decay as Form of Unsupervised Memory Resetting +--- + +In a natural, unsupervised setting, the object that a Monty system is observing will change from time to time. Currently, Monty's internal representations of objects is only explicitly reset by the experimenter, for example at the start of an episode of training. + +We would like to see if we can achieve reasonable performance when objects change (including during learning) by having a shorter memory horizon that rapidly decays. Assuming policies are sufficiently efficient in their exploration of objects, this should enable us to effectively determine whether we are still on the same object, on a different (but known) object, or on an entirely new object. This can subsequently inform changes such as switching to an exploration-focused policy (see [Implement Switching Between Learning and Inference-Focused Policies](../motor-system-improvements/implement-switching-between-learning-and-inference-focused-policies.md)). + +Note that we already have the `past_weight` and `present_weight` parameters, which can be used for this approach. As such, the main task is to set up experiments where objects are switched out without resetting the LMs evidence values, and then evaluate the performance of the system. + +If this fails to achieve the results we hope for, we might add a mechanism to explicitly reset evidence values when an LM believes it has moved on to a new object. In particular, we have implemented a method to detect when we have moved on to a new object based on significant changes in the accumulated evidence values for hypotheses. Integrating this method into the LMs is still in progress, but once complete, we would like to complement it with a process to reinitialize the evidence scores within the learning module. That way, when the LM detects it is on a new object, it can cleanly estimate what this new object might be. + +Eventually this could be complemented with top-down feedback from a higher-level LM modeling a scene or compositional object. In this case, the high-level LM biases the evidence values initialized in the low-level LM, based on what object should be present there according to the higher-level LM's model. Improvements here could also interact with the tasks of [Re-Anchor Hypotheses](../learning-module-improvements/re-anchor-hypotheses.md), and [Use Better Priors for Hypothesis Initialization](../learning-module-improvements/use-better-priors-for-hypothesis-initialization.md). diff --git a/docs/future-work/learning-module-improvements/reinitialize-hypotheses-when-starting-to-recognize-a-new-object.md b/docs/future-work/learning-module-improvements/reinitialize-hypotheses-when-starting-to-recognize-a-new-object.md deleted file mode 100644 index 0b79e216..00000000 --- a/docs/future-work/learning-module-improvements/reinitialize-hypotheses-when-starting-to-recognize-a-new-object.md +++ /dev/null @@ -1,7 +0,0 @@ ---- -title: Reinitialize Hypotheses When Starting to Recognize a new Object ---- - -We have recently implemented a method to detect when we have moved on to a new object based on significant changes in the accumulated evidence values for hypotheses. Integrating this method into the LMs is still in progress, but once complete, we would like to complement it with a process to reinitialize the evidence scores within the learning module. That way, when the LM detects it is on a new object, it can cleanly estimate what this new object might be. - -Eventually this could be complemented with top-down feedback from a higher-level LM modeling a scene or compositional object. In this case, the high-level LM biases the evidence values initialized in the low-level LM, based on what object should be present there according to the higher-level LM's model. Improvements here could also interact with the tasks of [Re-Anchor Hypotheses](../learning-module-improvements/re-anchor-hypotheses.md), and [Use Better Priors for Hypothesis Initialization](../learning-module-improvements/use-better-priors-for-hypothesis-initialization.md). diff --git a/docs/future-work/motor-system-improvements.md b/docs/future-work/motor-system-improvements.md index 7d8e89a6..b3259e9a 100644 --- a/docs/future-work/motor-system-improvements.md +++ b/docs/future-work/motor-system-improvements.md @@ -8,11 +8,13 @@ These are the things we would like to implement: - [Interpret goal states in motor system & switch policies](motor-system-improvements/interpret-goal-states-in-motor-system-switch-policies.md) #goalpolicy - [Implement switching between learning and inference-focused policies](motor-system-improvements/implement-switching-between-learning-and-inference-focused-policies.md) #learning - [Bottom-up exploration policy for surface agent](motor-system-improvements/bottom-up-exploration-policy-for-surface-agent.md) #learning -- [Top-down exploration policy](motor-system-improvements/top-down-exploration-policy.md) #learning #numsteps +- [Model-based exploration policy](motor-system-improvements/model-based-exploration-policy.md) #learning #numsteps - [Implement efficient saccades driven by model-free and model-based signals](motor-system-improvements/implement-efficient-saccades-driven-by-model-free-and-model-based-signals.md) #numsteps #multiobj - [Learn policy using RL and simplified action space](motor-system-improvements/learn-policy-using-rl.md) #numsteps #speed - [Decompose goals into subgoals & communicate](motor-system-improvements/decompose-goals-into-subgoals-communicate.md) #goalpolicy - [Reuse hypothesis testing policy target points](motor-system-improvements/reuse-hypothesis-testing-policy-target-points.md) #goalpolicy #numsteps - [Implement a simple cross-modal policy](motor-system-improvements/implement-a-simple-cross-modal-policy-for-sensory-guidance.md) #learning #multiobj #goalpolicy #numsteps +- [Model-based policy to recognize an object before moving onto a new object](motor-system-improvements/model-based-policy-to-recognize-an-object-before-moving-on-to-a-new-object.md) #multiobj #compositional +- [Policy to quickly move to a new object](motor-system-improvements/policy-to-quickly-move-to-a-new-object.md) #speed #multiobj #compositional Please see the [instructions here](project-roadmap.md#how-you-can-contribute) if you would like to tackle one of these tasks. \ No newline at end of file diff --git a/docs/future-work/motor-system-improvements/bottom-up-exploration-policy-for-surface-agent.md b/docs/future-work/motor-system-improvements/bottom-up-exploration-policy-for-surface-agent.md index 001dbb62..c62d5b6b 100644 --- a/docs/future-work/motor-system-improvements/bottom-up-exploration-policy-for-surface-agent.md +++ b/docs/future-work/motor-system-improvements/bottom-up-exploration-policy-for-surface-agent.md @@ -4,4 +4,4 @@ title: Bottom-Up Exploration Policy for Surface Agent For the distant agent, we have a policy specifically tailored to learning, the naive scan policy, which systematically explores the visible surface of an object. We would like a similar policy for the surface agent that systematically spirals or scans across the surface of an object, at least in a local area. -This would likely be complemented by [Top-Down Exploration Policies](top-down-exploration-policy.md). \ No newline at end of file +This would likely be complemented by [Model-Based Exploration Policies](model-based-exploration-policy.md). diff --git a/docs/future-work/motor-system-improvements/implement-efficient-saccades-driven-by-model-free-and-model-based-signals.md b/docs/future-work/motor-system-improvements/implement-efficient-saccades-driven-by-model-free-and-model-based-signals.md index 53fe4914..60639a4e 100644 --- a/docs/future-work/motor-system-improvements/implement-efficient-saccades-driven-by-model-free-and-model-based-signals.md +++ b/docs/future-work/motor-system-improvements/implement-efficient-saccades-driven-by-model-free-and-model-based-signals.md @@ -9,3 +9,8 @@ In the model-free case, salient information available in the view-finder could d In the model-based case, two primary settings should be considered: - A single LM has determined that the agent should move to a particular location in order to test a hypothesis, and it sends a goal-state that can be satisfied with a saccade, rather than the entire agent jumping/teleporting to a new location. For example, saccading to where the handle of a mug is believed to be will refute or confirm the current hypothesis. This is the more important/immediate use case. - Multiple LMs are present, including a smaller subset of more peripheral LMs. If one of these peripheral LMs observes something of interest, it can direct a goal-state to the motor system to perform a saccade such that a dense sub-set of LMs are able to visualize the object. This is analogous to cortical feedback bringing the fovea to an area of interest. + +Such policies are particularly important in an unsupervised setting, where we will want to more efficiently explore objects in order to rapidly determine their identity, given we have no supervised signal to tell us whether this is a familiar object, or an entirely new one. This will be compounded by the fact that [evidence for objects will rapidly decay](../learning-module-improvements/implement-and-test-rapid-evidence-decay-as-form-of-unsupervised-memory-resetting) in order to better support the unsupervised setting. + +Unlike the hypothesis-testing policy, we would specifically like to implement these as a saccade policy for the distant agent (i.e. no translation of the agent, only rotation), as this is a step towards an agent that can sample efficiently in the real world without having to physically translate sensors through space. Ideally, this would be implemented via a unified mechanism of specifying a location in e.g. ego-centric 3D space, and the policy determining the necessary rotation to focus at that point in space. + diff --git a/docs/future-work/motor-system-improvements/implement-switching-between-learning-and-inference-focused-policies.md b/docs/future-work/motor-system-improvements/implement-switching-between-learning-and-inference-focused-policies.md index acc0c976..89d065e1 100644 --- a/docs/future-work/motor-system-improvements/implement-switching-between-learning-and-inference-focused-policies.md +++ b/docs/future-work/motor-system-improvements/implement-switching-between-learning-and-inference-focused-policies.md @@ -6,4 +6,4 @@ Currently, a Monty system cannot flexibly switch between a learning-focused poli This would be a specific example of a more general mechanism for switching between different policies, as discussed in [Switching Policies via Goal States](interpret-goal-states-in-motor-system-switch-policies.md). -Similarly, an LM should be able to determine the most appropriate *model-based* policies to initialize, such as the hypothesis-testing policy vs. a [top-down exploration policy](top-down-exploration-policy.md). \ No newline at end of file +Similarly, an LM should be able to determine the most appropriate *model-based* policies to initialize, such as the hypothesis-testing policy vs. a [model-based exploration policy](model-based-exploration-policy.md). diff --git a/docs/future-work/motor-system-improvements/top-down-exploration-policy.md b/docs/future-work/motor-system-improvements/model-based-exploration-policy.md similarity index 64% rename from docs/future-work/motor-system-improvements/top-down-exploration-policy.md rename to docs/future-work/motor-system-improvements/model-based-exploration-policy.md index 9f39d448..a255b66d 100644 --- a/docs/future-work/motor-system-improvements/top-down-exploration-policy.md +++ b/docs/future-work/motor-system-improvements/model-based-exploration-policy.md @@ -1,9 +1,13 @@ --- -title: Top-Down Exploration Policy +title: Model-Based Exploration Policy --- During exploration/learning-focused movement, we do not make use of any model-based, top-down policies driven by LMs. Two approaches we would like to implement are: - A model-based policy that moves the sensors to areas that potentially represent the explored limits of an object. For example, if we've explored the surface of an object but not the entirety of it, then there will be points on the edge of the learned model with few neighboring points. Exploring in these regions is likely to efficiently uncover novel observations. Note that a "false-positive" for this heuristic is that thin objects like a wire or piece of paper will have such regions naturally at their edges, so it should only represent a bias in exploration, not a hard rule. - A model-based policy that spends more time exploring regions associated with high-frequency feature changes, or discriminative object-parts. For example, the spoon and fork in YCB are easy to confuse if the models of their heads are not sufficiently detailed. Two heuristics to support greater exploration in this area could include: - High-frequency changes in low-level features means we need a more detailed model of that part of the object. For example, the point-normals change frequently at the head of the fork, and so we likely need to explore it in more detail to develop a sufficiently descriptive model. The feature-change sensor-module is helpful for ensuring these observations are processed by the learning-module, but a modified policy would actually encourage more exploration in these regions. - Locations that are useful for distinguishing objects, such as the fork vs. spoon heads, are worth knowing in detail, because they define the differences between similar objects. These points correspond to those that are frequently tested by the hypothesis-testing policy (see [Reuse Hypothesis-Testing Policy Target Points](../motor-system-improvements/reuse-hypothesis-testing-policy-target-points.md)), and such stored locations can be leveraged to guide exploration. - - As we introduce hierarchy, it may be possible to unify these concepts under a single policy, i.e. where frequently changing features can either be at the sensory (low-level) input, or at the more abstract level of incoming sub-objects. \ No newline at end of file + - As we introduce hierarchy, it may be possible to unify these concepts under a single policy, i.e. where frequently changing features can either be at the sensory (low-level) input, or at the more abstract level of incoming sub-objects. + +Model-based exploration policies will be particularly important in an unsupervised setting. In particular, Monty will need to efficiently explore objects in order to recognize the commonality between different perspectives from which they are observed. For example, after learning a coffee mug from one view, and then seeing it from a different angle, Monty can move around to see that these are two views of the same object, and combine these into a single model. However, this recognition requires some overlap between the parts of the mug that have been see during the first exposure and what is observed on the second exposure. Learning the full structure of an object through active exploration should be easier than relying on separate, idealized views of the object (e.g. 14 rotations corresponding to the 6 faces and 8 corners of a cube), where the overlap between view points may be minimal. + +Such active exploration will also be important when dealing with compositional or multi-object settings, where establishing views of all sides of an object is not straightforward due to the presence of other objects or object-parts. \ No newline at end of file diff --git a/docs/future-work/motor-system-improvements/model-based-policy-to-recognize-an-object-before-moving-on-to-a-new-object.md b/docs/future-work/motor-system-improvements/model-based-policy-to-recognize-an-object-before-moving-on-to-a-new-object.md new file mode 100644 index 00000000..4a7b19d9 --- /dev/null +++ b/docs/future-work/motor-system-improvements/model-based-policy-to-recognize-an-object-before-moving-on-to-a-new-object.md @@ -0,0 +1,8 @@ +--- +title: Model-Based Policy to Recognize an Object Before Moving onto a New Object +--- + +When there are multiple objects in the world (including different parts of a compositional object), it is beneficial to recognize the object currently observed (i.e. converge to high confidence) before moving onto a new object. + +Such a policy could have different approaches, such as moving back if an LM believes it has moved onto a new object (reactive), or using the model of the most-likely-hypothesis to try to stay on the object (pro-active), or a mixture of these. In either case, these would be instances of model-based policies. + diff --git a/docs/future-work/motor-system-improvements/policy-to-quickly-move-to-a-new-object.md b/docs/future-work/motor-system-improvements/policy-to-quickly-move-to-a-new-object.md new file mode 100644 index 00000000..aeba071f --- /dev/null +++ b/docs/future-work/motor-system-improvements/policy-to-quickly-move-to-a-new-object.md @@ -0,0 +1,11 @@ +--- +title: Policy to quickly move to a new object +--- + +When exploring an environment with multiple objects (including components of a compositional object), it is beneficial to quickly move to a new object when the current one has been recognized, so as to rapidly build up a model of the outside world. + +It would be useful to have a policy that uses a mixture of model-free components (e.g. saliency map) and model-based components (learned relations of sub-objects to one another in a higher-level LM) to make a decision about where to move next in such an instance. + +This therefore relates to both [Model-based policy to recognize an object before moving on to a new object](./model-based-policy-to-recognize-an-object-before-moving-on-to-a-new-object.md) and [Implement efficient saccades driven by model-free and model-based signals](../motor-system-improvements/implement-efficient-saccades-driven-by-model-free-and-model-based-signals.md). + +Ideally, both this policy and the policy to remain on an object could be formulated together as a form of curiosity, where a learning module aims to reduce uncertainty about the world state. \ No newline at end of file diff --git a/docs/future-work/voting-improvements/generalize-voting-to-associative-connections.md b/docs/future-work/voting-improvements/generalize-voting-to-associative-connections.md index 9d8216b3..27320b32 100644 --- a/docs/future-work/voting-improvements/generalize-voting-to-associative-connections.md +++ b/docs/future-work/voting-improvements/generalize-voting-to-associative-connections.md @@ -5,6 +5,8 @@ Currently, voting relies on all learning modules sharing the same object ID for However, in the setting of unsupervised learning, the object ID that is associated with any given model is unique to the parent LM. As such, we need to organically learn the mapping between the object IDs that occur together across different LMs, such that voting can function without any supervised learning signal. This is the same issue faced by the brain, where a neural encoding in one cortical column (e.g. an SDR), needs to be associated with the different SDRs found in other cortical columns. +It is also worth noting that being able to use voting within unsupervised settings will enable us to converge faster, offsetting the issue of not knowing whether we have moved to a new object or not. This relates to the fact that [evidence for objects will rapidly decay](../learning-module-improvements/implement-and-test-rapid-evidence-decay-as-form-of-unsupervised-memory-resetting) in order to better support the unsupervised setting. + Initially, such voting would be explored within modality (two different vision-based LMs learning the same object), or across modalities with similar object structures (e.g. the 3D objects of vision and touch). However, this same approach should unlock important properties, such as associating models that may be structurally very different, like the vision-based object of a cow, and the auditory object of "moo" sounds. Furthermore, this should eventually enable associating learned words with grounded objects, laying the foundations for language. Finally, this challenge relates to [Use Pose for Voting](./use-pose-for-voting.md), where we would like to vote on the poses of objects, since the learned poses are also going to be unique to each LM. \ No newline at end of file diff --git a/docs/hierarchy.md b/docs/hierarchy.md index c90f16f7..b10666fd 100644 --- a/docs/hierarchy.md +++ b/docs/hierarchy.md @@ -80,7 +80,7 @@ - [learning-module-improvements](future-work/learning-module-improvements.md) - [contributing-learning-modules](future-work/learning-module-improvements/contributing-learning-modules.md) - [use-off-object-observations](future-work/learning-module-improvements/use-off-object-observations.md) - - [reinitialize-hypotheses-when-starting-to-recognize-a-new-object](future-work/learning-module-improvements/reinitialize-hypotheses-when-starting-to-recognize-a-new-object.md) + - [implement-and-test-rapid-evidence-decay-as-form-of-unsupervised-memory-resetting](future-work/learning-module-improvements/implement-and-test-rapid-evidence-decay-as-form-of-unsupervised-memory-resetting.md) - [improve-bounded-evidence-performance](future-work/learning-module-improvements/improve-bounded-evidence-performance.md) - [use-models-with-fewer-points](future-work/learning-module-improvements/use-models-with-fewer-points.md) - [make-it-possible-to-store-multiple-feature-maps-on-one-graph](future-work/learning-module-improvements/make-it-possible-to-store-multiple-feature-maps-on-one-graph.md) @@ -96,13 +96,15 @@ - [motor-system-improvements](future-work/motor-system-improvements.md) - [implement-switching-between-learning-and-inference-focused-policies](future-work/motor-system-improvements/implement-switching-between-learning-and-inference-focused-policies.md) - [bottom-up-exploration-policy-for-surface-agent](future-work/motor-system-improvements/bottom-up-exploration-policy-for-surface-agent.md) - - [top-down-exploration-policy](future-work/motor-system-improvements/top-down-exploration-policy.md) + - [model-based-exploration-policy](future-work/motor-system-improvements/model-based-exploration-policy.md) - [implement-efficient-saccades-driven-by-model-free-and-model-based-signals](future-work/motor-system-improvements/implement-efficient-saccades-driven-by-model-free-and-model-based-signals.md) - [learn-policy-using-rl](future-work/motor-system-improvements/learn-policy-using-rl.md) - [decompose-goals-into-subgoals-communicate](future-work/motor-system-improvements/decompose-goals-into-subgoals-communicate.md) - [interpret-goal-states-in-motor-system-switch-policies](future-work/motor-system-improvements/interpret-goal-states-in-motor-system-switch-policies.md) - [reuse-hypothesis-testing-policy-target-points](future-work/motor-system-improvements/reuse-hypothesis-testing-policy-target-points.md) - [implement-a-simple-cross-modal-policy-for-sensory-guidance](future-work/motor-system-improvements/implement-a-simple-cross-modal-policy-for-sensory-guidance.md) + - [model-based-policy-to-recognize-an-object-before-moving-on-to-a-new-object](future-work/motor-system-improvements/model-based-policy-to-recognize-an-object-before-moving-on-to-a-new-object.md) + - [policy-to-quickly-move-to-a-new-object](future-work/motor-system-improvements/policy-to-quickly-move-to-a-new-object.md) - [voting-improvements](future-work/voting-improvements.md) - [use-pose-for-voting](future-work/voting-improvements/use-pose-for-voting.md) - [outline-routing-protocol-attention](future-work/voting-improvements/outline-routing-protocol-attention.md)