Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v1.0.0 alpha #154

Draft
wants to merge 222 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
222 commits
Select commit Hold shift + click to select a range
12bdc1d
added learning for online variational filtering
dimarkov Jun 10, 2024
754a336
optimized learning routines to minimize calls to tree_map
dimarkov Jun 10, 2024
eda7e01
fix tests and passing of the results of the learning
dimarkov Jun 10, 2024
147a37c
fix update state transition
dimarkov Jun 10, 2024
bdf1e14
move task.py to jax/envs/env.py
Jun 10, 2024
71a3cb2
fix names
dimarkov Jun 10, 2024
65b33c8
notebook for showcasing inference and learning using different methods
dimarkov Jun 10, 2024
a82d7a1
Merge pull request #133 from infer-actively/pymdp-move-PymdpEnv
conorheins Jun 10, 2024
19172db
add a pyproject toml
OzanCatalVerses Jun 10, 2024
948409a
move task.py to jax/envs/env.py
Jun 10, 2024
a1bc4b0
get a version that passes all tests
OzanCatalVerses Jun 10, 2024
be0a8f5
Merge pull request #134 from infer-actively/feature/pyproject-toml
conorheins Jun 10, 2024
13ae015
added an example of A tensors and B tensors where smoothing should di…
conorheins Jun 10, 2024
ae528d9
did first step of inference in the ovf example
conorheins Jun 10, 2024
11a2f91
add a pyproject toml
OzanCatalVerses Jun 10, 2024
f9f9bab
get a version that passes all tests
OzanCatalVerses Jun 10, 2024
99f8253
add a pyproject toml
OzanCatalVerses Jun 10, 2024
a975e7d
get a version that passes all tests
OzanCatalVerses Jun 10, 2024
9db1556
initial graph worlds version
Jun 10, 2024
ad9c098
named distribution implementation
alec-tschantz Jun 11, 2024
6d83206
updated example
dimarkov Jun 11, 2024
b1e50d3
Merge branch 'full-ovf-with-example' into full-ovf-implementation
dimarkov Jun 11, 2024
97b3716
removed oudated markdown cell at the end of the smoothing example not…
conorheins Jun 11, 2024
980d86e
update graph worlds env
Jun 11, 2024
0e0e1f5
add a first version of model compilation
OzanCatalVerses Jun 11, 2024
9f17816
add missing s, add some basic tests
OzanCatalVerses Jun 11, 2024
630c59e
add unittest
OzanCatalVerses Jun 11, 2024
3091da7
rename compile to compile_model
OzanCatalVerses Jun 11, 2024
465bfac
added example notebook
alec-tschantz Jun 11, 2024
4c07332
add some documentation
OzanCatalVerses Jun 11, 2024
a97f5d3
api updates to jax/agent
alec-tschantz Jun 11, 2024
f55a0a8
Merge branch 'feature/distribution-api' of https://github.com/infer-a…
alec-tschantz Jun 11, 2024
beca262
add get_dependencies
OzanCatalVerses Jun 11, 2024
d070e37
clean up a bit
OzanCatalVerses Jun 11, 2024
3685a93
add dependency extraction to jax agent
alec-tschantz Jun 11, 2024
2fd13dc
acknowledge that the alphabet has an ordering
OzanCatalVerses Jun 11, 2024
f3bc163
end to end test of agent API
alec-tschantz Jun 11, 2024
49f10a6
forgot to push notebook
alec-tschantz Jun 11, 2024
5f40eb0
update random_B_matrix to handle complex action dependencies
ran-weii Jun 11, 2024
fb29fb3
change config naming to be more consistent
OzanCatalVerses Jun 11, 2024
3381edc
config naming changes
OzanCatalVerses Jun 11, 2024
8aa2295
make the data argument optional
OzanCatalVerses Jun 11, 2024
eab270d
updated agent api
alec-tschantz Jun 11, 2024
bc591ae
updated notebook
alec-tschantz Jun 11, 2024
df499f4
updated init for agent
alec-tschantz Jun 11, 2024
81ffea7
add utils to convert combinations to flattened index with test
ran-weii Jun 11, 2024
d7500d2
updated example notebook
alec-tschantz Jun 11, 2024
b3beca5
added complicated example
alec-tschantz Jun 11, 2024
7927648
Merge branch 'feature/distribution-api' into feature/complex_act_depe…
ran-weii Jun 11, 2024
a9b4404
add rollout function and fix graph world env and demo notebook
Jun 11, 2024
cf0058d
initial solution for flattening B action dims
ran-weii Jun 11, 2024
910dd89
move a copy of combination indexing utils to jax.utils
ran-weii Jun 11, 2024
614115d
initial solution for multi action decoding
ran-weii Jun 11, 2024
af08d88
vectorize combination indexing and update tests
ran-weii Jun 11, 2024
179317e
handle no action dependency in B tensor flattening, use vectorized mu…
ran-weii Jun 11, 2024
616fa59
commit complex_action_dependency notebook
ran-weii Jun 11, 2024
58ef4af
add knapsack_demo notebook, planning not working
ran-weii Jun 12, 2024
21e3b58
add function to construct flattend policies
ran-weii Jun 12, 2024
bb1ac49
commit working knapsack demo notebook
ran-weii Jun 12, 2024
4626c74
add some docstring
Jun 12, 2024
55144f9
add some more comments in the notebook
Jun 12, 2024
0856bce
smoothing demo notebook starting to work with sparse BCOO arrays from…
conorheins Jun 12, 2024
ccfe3d4
in `pymdp/jax/inference.py`, added version of joint_dist_factor (sub-…
conorheins Jun 12, 2024
84cdbf1
add multimethod to the dependencies
OzanCatalVerses Jun 12, 2024
c11d2d2
Merge pull request #135 from infer-actively/feature/graph_env
conorheins Jun 12, 2024
400fba2
named distribution implementation
alec-tschantz Jun 11, 2024
2692973
add a first version of model compilation
OzanCatalVerses Jun 11, 2024
ce89a0e
add missing s, add some basic tests
OzanCatalVerses Jun 11, 2024
a742691
add unittest
OzanCatalVerses Jun 11, 2024
98bd8c2
rename compile to compile_model
OzanCatalVerses Jun 11, 2024
e930eb7
added example notebook
alec-tschantz Jun 11, 2024
55f3141
api updates to jax/agent
alec-tschantz Jun 11, 2024
435dcb1
add some documentation
OzanCatalVerses Jun 11, 2024
91b1de7
add get_dependencies
OzanCatalVerses Jun 11, 2024
a2bfbda
clean up a bit
OzanCatalVerses Jun 11, 2024
ec97a13
add dependency extraction to jax agent
alec-tschantz Jun 11, 2024
0c63c36
acknowledge that the alphabet has an ordering
OzanCatalVerses Jun 11, 2024
296795d
end to end test of agent API
alec-tschantz Jun 11, 2024
ae878dc
forgot to push notebook
alec-tschantz Jun 11, 2024
45ae9a0
change config naming to be more consistent
OzanCatalVerses Jun 11, 2024
0a3194b
config naming changes
OzanCatalVerses Jun 11, 2024
b838199
make the data argument optional
OzanCatalVerses Jun 11, 2024
6b5e8cc
updated agent api
alec-tschantz Jun 11, 2024
ba55a07
updated notebook
alec-tschantz Jun 11, 2024
7f54738
updated init for agent
alec-tschantz Jun 11, 2024
ac8f198
updated example notebook
alec-tschantz Jun 11, 2024
ff6a18f
added complicated example
alec-tschantz Jun 11, 2024
98106a4
Merge branch 'feature/distribution-api' of https://github.com/infer-a…
alec-tschantz Jun 12, 2024
1d26079
updated so we do not add a batch by default in agent constructor
alec-tschantz Jun 12, 2024
fac44bd
fixed normalize
alec-tschantz Jun 12, 2024
b10e328
function to rol out using jax
toonvdm Jun 11, 2024
7524294
agent action perception loop
toonvdm Jun 11, 2024
592169b
Working state multiple cues
toonvdm Jun 11, 2024
7e809e3
Updated notebook
toonvdm Jun 11, 2024
3d5387b
made environment functional and added docstrings
toonvdm Jun 12, 2024
8bd70d8
intermediate state
toonvdm Jun 12, 2024
f96b00b
bug resolved demo
salvatomm Jun 12, 2024
751f918
Clean up notebook for the generalized tmaze
toonvdm Jun 12, 2024
9fbcae3
remove duplicate line
toonvdm Jun 12, 2024
30233a9
remove duplicate
toonvdm Jun 12, 2024
1352a6b
line length agent
toonvdm Jun 12, 2024
0ef6a39
undo linting issues with agent.py
toonvdm Jun 12, 2024
4ec0363
undo linting issues with agent.py
toonvdm Jun 12, 2024
89208bb
smoothing example notebook now working without batch dimensions
conorheins Jun 12, 2024
9fbab82
functional sparse array version of smoothing_ovf in inference.py (do…
conorheins Jun 12, 2024
88a7aab
rename B_dependencies back to B_factor_list to not break tests
ran-weii Jun 12, 2024
384548b
fixed num_states / num_obs constructors
alec-tschantz Jun 12, 2024
1ec7123
update multi action encoding and decoing with test
ran-weii Jun 12, 2024
cb7ac8f
merge with distribution_api branch and resolve conflict
ran-weii Jun 12, 2024
25ec151
Use new rollout signature
toonvdm Jun 13, 2024
727a75a
Merge pull request #137 from toonvdm/feature/agent-demo
dimarkov Jun 13, 2024
72c8510
Merge remote-tracking branch 'origin/pymdp-verses-hackathon' into fea…
conorheins Jun 13, 2024
01db186
Merge pull request #136 from infer-actively/feature/distribution-api
conorheins Jun 13, 2024
4ff50a5
Merge remote-tracking branch 'origin/pymdp-verses-hackathon' into HEAD
conorheins Jun 13, 2024
07f37da
Merge pull request #143 from infer-actively/feature/complex_act_depen…
conorheins Jun 13, 2024
f30bf28
Merge remote-tracking branch 'origin/pymdp-verses-hackathon' into fea…
conorheins Jun 13, 2024
449851e
don't overwrite C, D, E tensors if apply batch is True
toonvdm Jun 13, 2024
66c9301
updated poetry.lock
toonvdm Jun 13, 2024
c9493a2
beginning of unit test file for sparse tensors backend
conorheins Jun 13, 2024
54374ae
add sparse spm dot initial version
OzanCatalVerses Jun 13, 2024
42e46f3
fixes to agent.py in pymdp.jax to deal with:
conorheins Jun 13, 2024
01165b1
use apply_batch=False when constructing agent in generalized t maze d…
conorheins Jun 13, 2024
c31bcec
add a separate py file for tree searching
Jun 12, 2024
5a5800b
intermediate sharing some tree building
Jun 12, 2024
031ce57
update
Jun 12, 2024
f9c11a4
step function
alec-tschantz Jun 12, 2024
e8b63cf
vmap over policies
alec-tschantz Jun 12, 2024
9b94a39
integrated alec's new step function
Jun 12, 2024
8c46cd8
we have a tree
Jun 12, 2024
6cbfd06
added vmap over leaves
alec-tschantz Jun 12, 2024
247d711
merging sophisticated inference backprop
alec-tschantz Jun 13, 2024
a373439
vmap over observations
alec-tschantz Jun 13, 2024
639bf48
Merge pull request #144 from infer-actively/fix/error-c-tensor
conorheins Jun 13, 2024
cb49577
fix spm_dot_sparse dims dims modifications
OzanCatalVerses Jun 13, 2024
b1b74aa
add sophisticated tmaze example
Jun 13, 2024
59131c6
Unit test for ovf smoothing using sparse tensors
toonvdm Jun 14, 2024
0a3faae
updated docstring for todense hack
toonvdm Jun 14, 2024
a0967dd
get the inference methods comparison notebook to work with latest hac…
OzanCatalVerses Jun 14, 2024
2edabf6
benchmarking notebook
toonvdm Jun 14, 2024
37b253d
fix multimethod matching
OzanCatalVerses Jun 17, 2024
787fb00
change array->arraylike in all type signatures
OzanCatalVerses Jun 18, 2024
d21d8b0
unit tests for learning + B_dependencies in update_state_transition_d…
toonvdm Jun 21, 2024
b7bbbf9
added learning from main + added docstring + unit tested jax implemen…
toonvdm Jun 22, 2024
6b7d50f
selective update of factors for learning the transition parameters
toonvdm Jun 22, 2024
03b2cd1
return all transition matrices when factors_to_update is not all
toonvdm Jun 23, 2024
568900d
return all transition matrices when factors_to_update is not all
toonvdm Jun 23, 2024
02cd58c
remove legacy comment
toonvdm Jun 23, 2024
256f143
Merge pull request #148 from infer-actively/feature/learning-unit-tests
dimarkov Jun 27, 2024
2a1c34b
add imageio dependency
Jun 27, 2024
8d3bc80
also allow to use Distribution for C and D
Jun 27, 2024
2ad121f
Added Leaf class to register arbitrary objects as leaf
toonvdm Jun 27, 2024
92b5e4d
comment policies multi
toonvdm Jun 27, 2024
0cfc999
Resolve merge conflicts
toonvdm Jun 27, 2024
bee8872
add demo notebooks for the two jax envs
Jun 27, 2024
3ba3143
Merge branch 'feature/sophisticated-jax-rebased' into hackaton-demos
Jun 27, 2024
d1b5a0e
refactor si code and examples
Jun 27, 2024
c249dd5
move generate graph function to env file
Jun 27, 2024
2605d36
running sophisticated inference in a generalized tmaze env
dimarkov Jun 13, 2024
11a3e4d
add mctx based mcts planner
Jun 27, 2024
84c9943
simpler graph world demo
Jun 27, 2024
086b881
use custom scan to use scan signature to iterate over sparse arrays
toonvdm Jun 27, 2024
1b129bc
Merge branch 'feature/sparse_tensors_backend' into hackaton-demos
Jun 27, 2024
1273241
create separate subdir for the sparse experiment
Jun 27, 2024
092beed
Added mctx and mediapy dependencies
toonvdm Jun 28, 2024
927767e
generated outputs for benchmark notebook
toonvdm Jun 28, 2024
4da4476
add readable string repr for Distribution
tverbele Jul 31, 2024
bac99ed
add Model class for describing the full model in api
tverbele Jul 31, 2024
f91c1ab
fix bug in creating preference distribution
tverbele Aug 2, 2024
4dfe790
also allow specifying state preferences (H) through model/distribution
tverbele Aug 8, 2024
d7f605a
allow to set gamma and make default value 1
tverbele Aug 8, 2024
8bf9a38
use muzero_policy as default instead of the gumbel_muzero_policy
tverbele Aug 9, 2024
3f1287a
Merge pull request #149 from infer-actively/api_model
conorheins Sep 20, 2024
75123df
Merge pull request #151 from infer-actively/si_fixes
conorheins Sep 20, 2024
6a46032
Merged @dimarkov's upstream changes to `master` into `hackaton_demos`
conorheins Sep 20, 2024
707dd43
Merge pull request #152 from infer-actively/hackaton-demos
conorheins Sep 20, 2024
dfdd9d0
fix Distribution API notebook
Sep 20, 2024
908aa8a
fixed the initialization of the E vector in `Agent` constructor so th…
conorheins Sep 20, 2024
368907f
Added `apply_batch=False` flag to all initializations of an `Agent` i…
conorheins Sep 20, 2024
cb78e18
fix Graph Worlds demo
Sep 20, 2024
c9031f9
changes to `control.py`:
conorheins Sep 20, 2024
5a09229
fixed deprecated syntax of `Agent.infer_states` in `mcts.py` and `si.py`
conorheins Sep 20, 2024
c4e05ff
fixes to `sparse_benchmark.ipynb` notebook:
conorheins Sep 20, 2024
18aac74
cleanup the dependencies and poetry .toml
Sep 23, 2024
249db72
remove unused pandas import
Sep 23, 2024
a741886
fix distribution unit test
Sep 23, 2024
d9c9a83
fix update_empirical_prior and infer_states in tests
Sep 23, 2024
ff6e27a
configure setuptools using pyproject.toml
Sep 23, 2024
d41fccc
fix typo
Sep 23, 2024
e0d63c3
declare packages
tverbele Sep 24, 2024
f9049da
remove requirements.txt from README
tverbele Sep 24, 2024
5afbf38
added back in `decode_multi_actions` as method of `Agent`, originally…
conorheins Sep 24, 2024
c9630af
added optional `factors_to_update` argument into `pymdp.jax.learning.…
conorheins Sep 24, 2024
6652e39
fixed unit test `test_sparse_smoothing` in `test_jax_sparse_backend` …
conorheins Sep 24, 2024
89ac754
removed calls to `np.random.seed(0)` in `test_learning_jax.py`
conorheins Sep 24, 2024
58590de
added more authors to author list in `pyproject.toml` (unfinished/WIP)
conorheins Sep 24, 2024
265e50a
fixed `update_state_likelihood_dirichlet` to allow for `factors_to_up…
conorheins Sep 24, 2024
57d730d
Refactor of `pymdp` so that `jax` is the main backend, and numpy vers…
conorheins Sep 24, 2024
cc48366
fix refactored packages
Sep 24, 2024
d235ca6
removed `field(static=True)` option for `policies` matrix in `Agent`
conorheins Sep 24, 2024
b882927
reimplementation of missing line when checking `pA` in self.validate(…
conorheins Sep 24, 2024
fd44804
update to `pymdp.Agent`:
conorheins Sep 24, 2024
03991da
changed github workflow configuration .yml to no longer test on Pytho…
conorheins Sep 30, 2024
da3716c
- renamed `PyMDPEnv` to `Env` throughout the repo
conorheins Oct 2, 2024
2d9e755
renamed `PyMDPEnv` to `Env` in `learning_gridworld.ipynb` notebook
conorheins Oct 2, 2024
c3b795b
renamed `PyMDPEnv` to `Env` in `graph_worlds.py`
conorheins Oct 2, 2024
786ec50
renamed `PyMDPEnv` to `Env` in `grid_world_demo.ipynb`
conorheins Oct 2, 2024
b4513b4
started refactoring `GeneralizedTMaze(Env)` class
conorheins Oct 2, 2024
99cf06e
removed .jax module prefixes from imports in `pymdp.envs.rollout`
conorheins Oct 2, 2024
591bf24
refactor legacy TMaze to jax Env
Oct 2, 2024
696ffe9
add TMaze demo notebook
Oct 2, 2024
9733895
refactor PyMDPEnv to Env
Oct 2, 2024
18f6903
Merge branch 'tmaze' into v1.0.0_alpha
Oct 2, 2024
a958270
fix reset
Oct 2, 2024
82b80e5
add pymdp.envs.assets to package
Oct 3, 2024
0e644f1
package image files in assets
Oct 3, 2024
d081c23
fix state initialization on reset
Oct 3, 2024
6a0c2b0
flatten pB action dims for parameter learning with multi actions
ran-wei-verses Nov 18, 2024
fa5213a
Merge pull request #161 from ran-wei-verses/v1.0.0_alpha
conorheins Dec 2, 2024
b49587d
Set minimum Python version to 3.10 for improved compatibility and fea…
nikolamilovic1 Dec 7, 2024
66ae9f3
Add setup.cfg
nikolamilovic1 Dec 7, 2024
af28a0f
Modify setup.cfg to be equivalent to pyproject.toml.
nikolamilovic1 Dec 7, 2024
3f06266
Merge pull request #162 from nikolamilovic-ft/v1.0.0_alpha
conorheins Dec 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/python-package.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12"]
python-version: ["3.11", "3.12"]

steps:
- uses: actions/checkout@v2
Expand All @@ -27,11 +27,11 @@ jobs:
run: |
python -m pip install --upgrade pip
python -m pip install flake8 pytest
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
python -m pip install -e .
- name: Lint with flake8
run: |
# exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
run: |
pytest test
pytest test
3 changes: 1 addition & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,8 +120,7 @@ If you would like to contribute to this repo, we recommend using venv and pip
cd <path_to_repo_fork>
python3 -m venv env
source env/bin/activate
pip install -r requirements.txt
pip install -e ./ # This will install pymdp as a local dev package
pip install -e . # This will install pymdp as a local dev package
```

You should then be able to run tests locally with `pytest`
Expand Down
2 changes: 1 addition & 1 deletion docs/env.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,6 @@ same general usage as above.

pymdp.envs.GridWorldEnv
pymdp.envs.DGridWorldEnv
pymdp.envs.VisualForagingEnv
pymdp.envs.SceneConstruction
pymdp.envs.TMazeEnv
pymdp.envs.TMazeEnvNullOutcome
261 changes: 0 additions & 261 deletions examples/A_matrix_demo.ipynb

This file was deleted.

Loading