Release v0.3 (#63)

* colab links in readme and docs * move colab link to getting_started in docs, add colab banner in readme * basic support for logging warnings * wandb logging; revamped warning logging; macros * addressed PR comments; moved wandb proj name to experiments.logging section; hyperparam generator populates tags * change tags to meta. add wandb api key to macros * check that Macros.WANDB_ENTITY is not None * fix depth unprocessing, fix config modality handling * limit torch threads * space saving options in dataset extraction script * changes to make image uint8 to float conversion happen on GPU * disable load next obs if BC * small bug fix * add storage flags to docs * add some 2022 refs * add viola * add link to old docs on tab bar and logo into tab bar * make sure kwargs are empty in default configs for easy override * move some models into obs core file * minor change * work with older robosuite versions * add some 2023 refs * making validation hdf5 key explicit * IQL implementation; todo: comments and testing * revamped comments for IQL * add minimal iql test and fix template * added comments and tests; removed filter_type feature * updated docs and configs for d4rl v2 envs; added iql results * update iql template json config to reflect v0.3 changes * generate iql configs for d4rl * fixed small bug with d4rl output dirs * update version numbers for d4rl docs * adding r3m and mvp pretraining * clean * add doc for pretrain * fix * revert change to env robosuite * minor rename * update pretrained reps tutorial * fix minor issues * small update to iql config * small docs fix * add orbit reference * more 2023 papers * working version of color jitter net * add gaussian noise randomizer * fix bug in GaussianNoiseRandomizer * randomizer visualization * center cropping for CropRandomizer eval mode * viz randomizer * move randomizers to obs_core.py * remove dups * remove dups * missing import * doc instruction for visualizing randomizer * Fix tab issue * update reqs with matplotlib * modified docs for new datasets, new algos, wandb support * add note about using offline_study datasets * postprocess xml for v1.3 * update postprocess function * added verification exps * limit num torch threads * updated dataset gen script * mj211 verification runs * started conversion script for robosuite v1.3.1. WIP * initial conversion script ready * add verion to env_robosuite; fix bug from observation modality PR * added v_1_3_1 verification jsons * converted lift dataset, wip on can * playback with can appears working but state divergence still high * all tasks minimally in place now without mujoco exceptions * minimal script for robosuite v140 conversion; created configs * updated robosuite v140 convertion script * added version checks for robosuite * update dataset registry to point to v141 raw datasets * generating and downloading datasets for v141 * updated obs extraction script to use v141 robosuite datasets * remove unsued scripts * remove unnec script * delete old verification scripts * remove unnec changes * add low_dim dataset links back in, more informative message for unstored datasets * Transformer support (#38) * initial BC-mingpt implementation * added adamw and weight scheduler * updated mingpt configs for sweep * rename mingpt -> transformer * cleaned up bc transformer in bc.py * cleaned up code comments * remove internal config gen scripts * addressing Danfei's comments in PR * update template configs and make some cosmetic changes * add ack for mingpt --------- Co-authored-by: Ajay Mandlekar <[email protected]> Co-authored-by: amandlek <[email protected]> * bump version and minor docs update * remove file * some minor docs fixes for updated test_v141 hdf5 * small change to gitignore for test hdf5 * some minor docs fixes * update 2023 refs * Update obs_core.py with import * Add wandb description in "viewing results" tutorial page * added more wandb info to docs * fix observation postprocessing for nested obs dicts * added bc transformer tests * V0.3 transformer tutorial (#59) * added transformer tutorial * Update bc_transformer.json * Update bc_transformer.json * Update training_transformers.md * adjusting tutorial docs based on feedback * changes to pretrained representations and transformer tutorial * added data loader params for pretrained reps tutorial --------- Co-authored-by: Bronars <[email protected]> * add readme note * V0.3 pretrained rep fix (#64) * fixed error in base_nets for R3MConv * added error messages for Sequential * added doc string for Sequential init --------- Co-authored-by: Danfei Xu <[email protected]> Co-authored-by: snasiriany <[email protected]> Co-authored-by: josiah_wong <[email protected]> Co-authored-by: Soroush Nasiriany <[email protected]> Co-authored-by: Soroush Nasiriany <[email protected]> Co-authored-by: j96w <[email protected]> Co-authored-by: danfei <[email protected]> Co-authored-by: MBronars <[email protected]> Co-authored-by: Bronars <[email protected]>
ARISE-Initiative · Jul 4, 2023 · 40e427a · 40e427a
1 parent b5d2aa9
commit 40e427a
Show file tree

Hide file tree

Showing 98 changed files with 4,997 additions and 1,707 deletions.
diff --git a/.gitignore b/.gitignore
@@ -6,6 +6,7 @@ dist/
 
 # local test dataset that is lazily downloaded by example scripts
 tests/assets/test.hdf5
+tests/assets/test_v141.hdf5
 
 # pycharm configs
 .idea/
@@ -119,3 +120,6 @@ venv.bak/
 .mypy_cache/
 *.mp4
 *.pth
+
+# private macros
+macros_private.py
diff --git a/README.md b/README.md
@@ -15,10 +15,19 @@
 
 -------
 ## Latest Updates
+- [07/03/2023] **v0.3**: BC-Transformer and IQL :brain:, support for DeepMind MuJoCo bindings :robot:, pre-trained image reps :eye:, wandb logging :chart_with_upwards_trend:, and more
 - [05/23/2022] **v0.2.1**: Updated website and documentation to feature more tutorials :notebook_with_decorative_cover:
-- [12/16/2021] **v0.2.0**: Modular observation modalities and encoders :wrench:, support for [MOMART](https://sites.google.com/view/il-for-mm/home) datasets :open_file_folder:
+- [12/16/2021] **v0.2.0**: Modular observation modalities and encoders :wrench:, support for [MOMART](https://sites.google.com/view/il-for-mm/home) datasets :open_file_folder: [[release notes]](https://github.com/ARISE-Initiative/robomimic/releases/tag/v0.2.0) [[documentation]](https://robomimic.github.io/docs/v0.2/introduction/overview.html)
 - [08/09/2021] **v0.1.0**: Initial code and paper release
 
+-------
+
+## Colab quickstart
+Get started with a quick colab notebook demo of robomimic with installing anything locally.
+
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1b62r_km9pP40fKF0cBdpdTO2P_2eIbC6?usp=sharing)
+
+
 -------
 
 **robomimic** is a framework for robot learning from demonstration.

diff --git a/docs/api/robomimic.algo.rst b/docs/api/robomimic.algo.rst
@@ -52,6 +52,14 @@ robomimic.algo.hbc module
    :undoc-members:
    :show-inheritance:
 
+robomimic.algo.iql module
+-------------------------
+
+.. automodule:: robomimic.algo.iql
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 robomimic.algo.iris module
 --------------------------
 

diff --git a/docs/api/robomimic.config.rst b/docs/api/robomimic.config.rst
@@ -60,6 +60,14 @@ robomimic.config.hbc\_config module
    :undoc-members:
    :show-inheritance:
 
+robomimic.config.iql\_config module
+-----------------------------------
+
+.. automodule:: robomimic.config.iql_config
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 robomimic.config.iris\_config module
 ------------------------------------
 

diff --git a/docs/api/robomimic.envs.rst b/docs/api/robomimic.envs.rst
@@ -36,6 +36,14 @@ robomimic.envs.env\_robosuite module
    :undoc-members:
    :show-inheritance:
 
+robomimic.envs.wrappers module
+------------------------------
+
+.. automodule:: robomimic.envs.wrappers
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 Module contents
 ---------------
 

diff --git a/docs/api/robomimic.models.rst b/docs/api/robomimic.models.rst
@@ -20,6 +20,14 @@ robomimic.models.distributions module
    :undoc-members:
    :show-inheritance:
 
+robomimic.models.obs\_core module
+---------------------------------
+
+.. automodule:: robomimic.models.obs_core
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 robomimic.models.obs\_nets module
 ---------------------------------
 
@@ -36,6 +44,14 @@ robomimic.models.policy\_nets module
    :undoc-members:
    :show-inheritance:
 
+robomimic.models.transformers module
+------------------------------------
+
+.. automodule:: robomimic.models.transformers
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 robomimic.models.vae\_nets module
 ---------------------------------
 

diff --git a/docs/api/robomimic.rst b/docs/api/robomimic.rst
@@ -13,6 +13,17 @@ Subpackages
    robomimic.models
    robomimic.utils
 
+Submodules
+----------
+
+robomimic.macros module
+-----------------------
+
+.. automodule:: robomimic.macros
+   :members:
+   :undoc-members:
+   :show-inheritance:
+
 Module contents
 ---------------
 

diff --git a/docs/api/robomimic.utils.rst b/docs/api/robomimic.utils.rst
@@ -52,14 +52,6 @@ robomimic.utils.loss\_utils module
    :undoc-members:
    :show-inheritance:
 
-robomimic.utils.macros module
------------------------------
-
-.. automodule:: robomimic.utils.macros
-   :members:
-   :undoc-members:
-   :show-inheritance:
-
 robomimic.utils.obs\_utils module
 ---------------------------------
 

diff --git a/docs/conf.py b/docs/conf.py
@@ -59,17 +59,17 @@
 
 # General information about the project.
 project = 'robomimic'
-copyright = '2022, Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang'
-author = 'Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang'
+copyright = 'the robomimic core team, 2023'
+author = 'the robomimic core team'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
 # built documents.
 #
 # The short X.Y version.
-version = robomimic.__version__
+version = (".").join(robomimic.__version__.split(".")[:-1])
 # The full version, including alpha/beta/rc tags.
-release = robomimic.__version__
+release = (".").join(robomimic.__version__.split(".")[:-1])
 
 
 # The language for content autogenerated by Sphinx. Refer to documentation
@@ -98,6 +98,7 @@
 # a list of builtin themes.
 #
 html_theme = 'sphinx_book_theme'
+html_logo = "robomimic_logo.png"
 
 # Theme options are theme-specific and customize the look and feel of a theme
 # further.  For a list of options available for each theme, see the

diff --git a/docs/datasets/d4rl.md b/docs/datasets/d4rl.md
@@ -9,9 +9,9 @@ Use `convert_d4rl.py` in the `scripts/conversion` folder to automatically downlo
 
 ```sh
 # by default, download to robomimic/datasets
-$ python convert_d4rl.py --env walker2d-medium-expert-v0
+$ python convert_d4rl.py --env walker2d-medium-expert-v2
 # download to specific folder
-$ python convert_d4rl.py --env walker2d-medium-expert-v0 --folder /path/to/output/folder/
+$ python convert_d4rl.py --env walker2d-medium-expert-v2 --folder /path/to/output/folder/
 ```
 
 - `--env` specifies the dataset to download
@@ -25,19 +25,19 @@ No postprocessing is required, assuming the above script is run!
 
 ## D4RL Results
 
-Below, we provide a table of results on common D4RL datasets using the algorithms included in the released codebase. We follow the convention in the TD3-BC paper, where we average results over the final 10 rollout evaluations, but we use 50 rollouts instead of 10 for each evaluation. Apart from a small handful of the halfcheetah results, the results align with those presented in the [TD3_BC paper](https://arxiv.org/abs/2106.06860). We suspect the halfcheetah results are different because we used `mujoco-py` version `2.0.2.13` in our evaluations, as opposed to `1.5` in order to be consistent with the version we were using for robosuite datasets. The results below were generated with `gym` version `0.17.3` and this `d4rl` [commit](https://github.com/rail-berkeley/d4rl/tree/9b68f31bab6a8546edfb28ff0bd9d5916c62fd1f).
+Below, we provide a table of results on common D4RL datasets using the algorithms included in the released codebase. We follow the convention in the TD3-BC paper, where we average results over the final 10 rollout evaluations, but we use 50 rollouts instead of 10 for each evaluation. All results are reported on the `-v2` environment variants. Apart from a small handful of the halfcheetah results, the results align with those presented in the [TD3_BC paper](https://arxiv.org/abs/2106.06860). We suspect the halfcheetah results are different because we used `mujoco-py` version `2.1.2.14` in our evaluations, as opposed to `1.5` in order to be consistent with the version we were using for robosuite datasets. The results below were generated with `gym` version `0.24.1` and this `d4rl` [commit](https://github.com/Farama-Foundation/D4RL/tree/305676ebb2e26582d50c6518c8df39fd52dea587).
 
-|                               | **BCQ**       | **CQL**       | **TD3-BC**    |
-| ----------------------------- | ------------- | ------------- | ------------- |
-| **HalfCheetah-Medium**        | 40.8% (4791)  | 38.5% (4497)  | 41.7% (4902)  |
-| **Hopper-Medium**             | 36.9% (1181)  | 30.7% (980)   | 97.9% (3167)  |
-| **Walker2d-Medium**           | 66.4% (3050)  | 65.2% (2996)  | 77.0% (3537)  |
-| **HalfCheetah-Medium-Expert** | 74.9% (9016)  | 21.5% (2389)  | 79.4% (9578)  |
-| **Hopper-Medium-Expert**      | 83.8% (2708)  | 111.7% (3614) | 112.2% (3631) |
-| **Walker2d-Medium-Expert**    | 70.2% (3224)  | 77.4% (3554)  | 102.0% (4683) |
-| **HalfCheetah-Expert**        | 94.3% (11427) | 29.2% (3342)  | 95.4% (11569) |
-| **Hopper-Expert**             | 104.7% (3389) | 111.8% (3619) | 112.2% (3633) |
-| **Walker2d-Expert**           | 80.5% (3699)  | 108.0% (4958) | 105.3% (4837) |
+|                               | **BCQ**       | **CQL**       | **TD3-BC**    | **IQL**       |
+| ----------------------------- | ------------- | ------------- | ------------- | ------------- |
+| **HalfCheetah-Medium**        | 46.8% (5535)  | 46.7% (5516)  | 47.9% (5664)  | 45.6% (5379)  |
+| **Hopper-Medium**             | 63.9% (2059)  | 59.2% (1908)  | 61.0% (1965)  | 53.7% (1729)  |
+| **Walker2d-Medium**           | 74.6% (3426)  | 79.7% (3659)  | 82.9% (3806)  | 77.0% (3537)  |
+| **HalfCheetah-Medium-Expert** | 89.9% (10875) | 77.6% (9358)  | 92.1% (11154) | 89.0% (10773) |
+| **Hopper-Medium-Expert**      | 79.5% (2566)  | 62.9% (2027)  | 89.7% (2900)  | 110.1% (3564) |
+| **Walker2d-Medium-Expert**    | 98.7% (4535)  | 109.0% (5007) | 111.1% (5103) | 109.7% (5037) |
+| **HalfCheetah-Expert**        | 92.9% (11249) | 67.7% (8126)  | 94.6% (11469) | 93.3% (11304) |
+| **Hopper-Expert**             | 92.3% (2984)  | 104.2% (3370) | 108.5% (3512) | 110.5% (3577) |
+| **Walker2d-Expert**           | 108.6% (4987) | 108.5% (4983) | 110.3% (5066) | 109.1% (5008) |
 
 
 ### Reproducing D4RL Results
@@ -52,4 +52,4 @@ In order to reproduce the results above, first make sure that the `generate_pape
   journal={arXiv preprint arXiv:2004.07219},
   year={2020}
 }
-```
+```
diff --git a/docs/datasets/overview.md b/docs/datasets/overview.md
@@ -159,8 +159,6 @@ You can easily list the filter keys present in a dataset with the `get_dataset_i
 
 </div>
 
-Using filter keys during training is easy. To use the generated train-valid split, you can set `config.experiment.validate=True` so that the demos under `mask/train` are used for training, and the demos under `mask/valid` are used for validation. 
-
-You can also use a custom filter key for training by setting `config.train.hdf5_filter_key=<FILTER_KEY_NAME>`. This ensures that only the demos under `mask/<FILTER_KEY_NAME>` are used during training. If you also set `config.experiment.validate=True`, this filter key's train-valid split will be used.
-
+Using filter keys during training is easy. To use the generated train-valid split, you can set `config.experiment.validate=True` to ensure that validation will run after each training epoch, and then set `config.train.hdf5_filter_key="train"` and `config.train.hdf5_validation_filter_key="valid"` so that the demos under `mask/train` are used for training, and the demos under `mask/valid` are used for validation. 
 
+You can also use a custom filter key for training by setting `config.train.hdf5_filter_key=<FILTER_KEY_NAME>`. This ensures that only the demos under `mask/<FILTER_KEY_NAME>` are used during training. You can also specify a custom filter key for validation by setting `config.train.hdf5_validation_filter_key`.