Skip to content

Commit

Permalink
ran through demos to verify they work with latest changes
Browse files Browse the repository at this point in the history
  • Loading branch information
@conorheins
conorheins committed Oct 27, 2021
1 parent 8104f42 commit bcb7440
Show file tree
Hide file tree
Showing 11 changed files with 1,041 additions and 3,567 deletions.
126 changes: 54 additions & 72 deletions examples/A_matrix_demo.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,26 @@
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Generative Model Demo: Constructing a simple likelihood model \n",
"This demo notebook provides a walk-through of how to build a simple A matrix (or likelihood mapping) that encodes an aegnt's beliefs about how hidden states 'cause' or probabilistically relate to observations"
],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Imports\n",
"\n",
"First, import `pymdp` and the modules we'll need."
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": 21,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"import sys\n",
Expand All @@ -36,12 +38,11 @@
"import pymdp.utils as utils\n",
"from pymdp.utils import create_A_matrix_stub, read_A_matrix\n",
"from pymdp.algos import run_fpi"
],
"outputs": [],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## The world (as represented by the agent's generative model)\n",
"\n",
Expand All @@ -68,12 +69,13 @@
"#### 2. `WEATHER_OBSERVATION`\n",
"\n",
"The second modality is a ternary (3-valued) variable representing the agent's observation of the state of the weather, e.g. by looking at the sky. In this example, it can either look `clear`, `rainy`, or `cloudy`\n"
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": 22,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"model_labels = {\n",
" \"observations\": {\n",
Expand All @@ -99,20 +101,20 @@
"pre_specified_excel = True\n",
"\n",
"A_stub = create_A_matrix_stub(model_labels)"
],
"outputs": [],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Option 1. Write the empty A matrix stub to an excel file, fill it out separately (e.g. manually in excel, and then read it back into memory). Remember, these represent the agent's generative model, not the true probabilities that relate states to observations. So you can think of these as the agent's personal/subjective 'assumptions' about how hidden states relate to observations."
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": 23,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"if read_from_excel:\n",
" ## Option 1: fill out A matrix 'offline' (e.g. in an excel spreadsheet)\n",
Expand All @@ -126,37 +128,37 @@
" if not pre_specified_excel:\n",
" A_stub.to_excel(excel_path)\n",
" print(f'Go fill out the A matrix in {excel_path} and then continue running this code\\n')"
],
"outputs": [],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"After you've filled out the Excel sheet separately (e.g. opening up Microsoft Excel and filling out the cells, you can read it back into memory)"
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": 24,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"if read_from_excel:\n",
" A_stub = read_A_matrix(excel_path, n_factors)"
],
"outputs": [],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Option 2. Fill out the A matrix using the desired probabilities. Remember, these represent the agent's generative model, not the true probabilities that relate states to observations. So you can think of these as the agent's personal/subjective 'assumptions' about how hidden states relate to observations."
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": 25,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"if not read_from_excel:\n",
" A_stub.loc[('grass_observation','wet'),('rained', 'on')] = 1.0\n",
Expand All @@ -172,36 +174,36 @@
" A_stub.loc['weather_observation','rained'] = np.tile(np.array([0.1, 0.65, 0.25]).reshape(-1,1), (1,2)) \n",
"\n",
" A_stub.loc[('weather_observation'),('did_not_rain')] = np.tile(np.array([0.9, 0.05, 0.05]).reshape(-1,1), (1,2)) \n"
],
"outputs": [],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Now we can use a utility function `convert_stub_to_ndarray` to convert the human-readable A matrix into the multi-dimensional tensor form needed by `pymdp` to achieve things like inference and action selection"
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": 26,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"A = utils.convert_A_stub_to_ndarray(A_stub, model_labels)"
],
"outputs": [],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Sample a random observation"
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": 34,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"obs_idx = [np.random.randint(o_dim) for o_dim in num_obs]\n",
"# obs_idx = [0, 1] # wet and rainy\n",
Expand All @@ -211,55 +213,35 @@
"for g, modality_name in enumerate(model_labels['observations'].keys()):\n",
" observation[g][obs_idx[g]] = 1.0\n",
" print('%s: %s'%(modality_name, model_labels['observations'][modality_name][obs_idx[g]]))"
],
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"grass_observation: dry\n",
"weather_observation: clear\n"
]
}
],
"metadata": {}
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Given the observation and your A matrix, perform inference to optimize a simple posterior belief about the state of the world "
],
"metadata": {}
]
},
{
"cell_type": "code",
"execution_count": 35,
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"qs = run_fpi(A, observation, num_obs, n_states, prior=None, num_iter=10, dF=1.0, dF_tol=0.001)\n",
"\n",
"print('Belief that it rained: %.2f'%(qs[0][0]))\n",
"print('Belief that the sprinkler was on: %.2f'%(qs[1][0]))"
],
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Belief that it rained: 0.00\n",
"Belief that the sprinkler was on: 0.33\n"
]
}
],
"metadata": {}
]
}
],
"metadata": {
"interpreter": {
"hash": "4144ffdb2ef81a5a74efd1d0bf9253d3fe81ec80475fd1008a839668d590b806"
"hash": "43ee964e2ad3601b7244370fb08e7f23a81bd2f0e3c87ee41227da88c57ff102"
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3.7.10 64-bit ('pymdp_env': conda)"
"display_name": "Python 3.7.10 64-bit ('pymdp_env': conda)",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
Expand All @@ -276,4 +258,4 @@
},
"nbformat": 4,
"nbformat_minor": 2
}
}
4 changes: 3 additions & 1 deletion examples/A_matrix_demo.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@
if read_from_excel:
## Option 1: fill out A matrix 'offline' (e.g. in an excel spreadsheet)

excel_dir = 'tmp_dir'
excel_dir = 'examples/tmp_dir'
if not os.path.exists(excel_dir):
os.mkdir(excel_dir)

Expand Down Expand Up @@ -93,3 +93,5 @@

print('Belief that it rained: %.2f'%(qs[0][0]))
print('Belief that the sprinkler was on: %.2f'%(qs[1][0]))

# %%
Loading

0 comments on commit bcb7440

Please sign in to comment.