STM tutorial as a POMDP

[IsobelWeb]

Can anyone give more detail on how the STM single sensor tutorial models sensor management as a POMDP? As far as I can tell, by 'assuming a predicted measurement corresponding to that [track] prediction' renders the whole thing deterministic. It reads to me that the Kalman Filter is a POMDP but the sensor management is an optimisation algorithm based on a deterministic collection of actions. Where's the stochasticity in the sensor management algorithm?

[nperree-dstl]

From my understanding, parts of the POMDP are similar to steps within a Kalman filter such as having an estimated state, some transition model for how the state evolves over time, and a measurement model for how the sensor observes the state. The sensor management provides the decision part of the POMDP.

You are correct that in Stone Soup the SensorManager class does contain the code for the chosen optimisation approach but it's primary purpose is to use a RewardFunction (Rk) to evaluate a set of actions (Ak) which could be taken by the sensors provided.

To quantify this reward the SensorManager also requires understanding of the state at time k which is provided by the current "tracks", as well as a predictor and updater to provide the transition and measurement models to use. With all this information a chosen action(s) can be selected by the SensorManager to meet some objective as described in the RewardFunction.

In case it's useful, the representation of sensor management as a POMDP is based on the problem described in Chapter 2 of the book "Foundations and Applications of Sensor Management" (Hero et al.), which may make more sense than my description.

[jmbarr]

In case it's useful, the representation of sensor management as a POMDP is based on the problem described in Chapter 2 of the book "Foundations and Applications of Sensor Management" (Hero et al.), which may make more sense than my description.

If you haven't got access to the book, there's a resource on the web which gives a good overview: http://web.eecs.umich.edu/~hero/Preprints/PartiallyObservable.pdf

[jmbarr]

Where's the stochasticity in the sensor management algorithm?

I don't think it's claimed that the sensor management algorithm is stochastic. The brute force sensor management algorithm uses an estimate of the current state of the system to make a prediction upon which to determine an optimal action. Given the same predicted states, the same optimal action will be chosen. (If I have the tutorial in question correct, there is a 'stochastic' sensor manager, but it's the random one, which is shown to be poorer.)

For a POMDP more generally, randomness enters at two points. Firstly is the transition from state $\mathbf{x}_{k-1}$ to state $\mathbf{x}_k$ (embodied in the transition_model), the second is the observation, since one must derive a measurement $\mathbf{z}_k$ from $\mathbf{x}_k$ (measurement_model). So even though the optimisation used within a sensor manager may be deterministic it is responding to a situation with random inputs. Hence if you run it many times over, you'll get different optimal actions returned. You may be able to show this by adjusting the random seed in the tutorial.