Is it possible to do model fitting using this package?

[yaowang1111]

Hi~

I want to know if this package can fit model to behavior datas collected by my experiment?
Or will there be future plans to implement this functionality?

Thanks~

[conorheins]

Hi @yaowang1111 ,
Thanks for your question -- unfortunately we can't do model-fitting yet, but this is high on the priority list. We are actively working on a new backend for pymdp that would allow us to fit the parameters of active inference POMDP agents using inference techniques from Numpyro, and to take advantage of GPU acceleration as well. I can't give you a timeline on this project at the moment, but I'll leave this issue open and update you with more information, once this functionality becomes available.

[yaowang1111]

Actually I'm woking on an active inference simulation project using pyro. I think Active inference shouldn't be confined to discrete matrix situations. Generalising it to any general probablistic models would benefit the theory. And pyro seems to be a suitable tool.
I have some very basic understanding of pyro now.
Can i somehow participate in this new version, despite of my limited coding skills?

[conorheins]

Hi @yaowang1111, sorry for the late reply on this! That sounds really cool, namely doing active inference in pyro.

It would be great to discuss more the involvement in the numpyro version of the package. At the moment this is a side-project for both myself and the other person (@dimarkov) working on this project, so we haven't had a chance to work on it in a while. Once we've fired that branch back up I'll get back and touch and we can catch you up on what we've done so far

[yaowang1111]

That would be great!

[conorheins]

Hi @yaowang1111 , just getting back in touch to let you know we've restarted our work on the numpyro-ified version of pymdp. The first step towards doing this was to reimplement all the core functions of pymdp in jax, so that they are autodifferentiable and we can pass gradients through the Agent and associated likelihood function.

Theagent_jax branch is the place to start, if you're interested in seeing what we've already done / contributing yourself.

In particular, see the model_inversion_demo.ipynb notebook, which shows the overall flow of how we'd write down a numpyro model for inverting a pymdp model. Cell 29 is when the pymdp active inference loop is written into a numpyro/pyro like model specification.

Also, the jax re-implementation of most of the relevant sub-functions in pymdp can be found in thepymdp/pymdp/jax subfolder of the agent_jax branch.

With respect to contributing/getting involved: There are several things that need to be done (and can be done in parallel), in case you're interested or have time to contribute. Let me know your thoughts / if you have time or interest in getting involved :) Also open to discussing what you've been doing in pyro and seeing how there might be overlap between these projects.

[conorheins]

I'm going to move this to a discussion since it's more relevant there.

[smonfas]

Hello,

I'm trying to do model inversion for a simple POMDP in my masters thesis, but I'm having some troubles with the jupyter notebook you provided above. Specifically, I can't set up the AIFA-Agent because it lacks the pA and pB arguments, which are not defined in the script. Am I right to assume, that these are the dirichlets for learning A and B? If I set learn_A and learn_B to False, I still get an error for not providing pA and pB. I tried creating the dirichlets with utils and jax.utils, but that doesn't seem to work either.

Any help is appreciated, thanks a lot!