From 40e427a695a9f085d3fc0f70ffb0d687a10802c8 Mon Sep 17 00:00:00 2001
From: amandlek
Date: Mon, 3 Jul 2023 17:23:13 -0700
Subject: [PATCH] Release v0.3 (#63)
* colab links in readme and docs
* move colab link to getting_started in docs, add colab banner in readme
* basic support for logging warnings
* wandb logging; revamped warning logging; macros
* addressed PR comments; moved wandb proj name to experiments.logging section; hyperparam generator populates tags
* change tags to meta. add wandb api key to macros
* check that Macros.WANDB_ENTITY is not None
* fix depth unprocessing, fix config modality handling
* limit torch threads
* space saving options in dataset extraction script
* changes to make image uint8 to float conversion happen on GPU
* disable load next obs if BC
* small bug fix
* add storage flags to docs
* add some 2022 refs
* add viola
* add link to old docs on tab bar and logo into tab bar
* make sure kwargs are empty in default configs for easy override
* move some models into obs core file
* minor change
* work with older robosuite versions
* add some 2023 refs
* making validation hdf5 key explicit
* IQL implementation; todo: comments and testing
* revamped comments for IQL
* add minimal iql test and fix template
* added comments and tests; removed filter_type feature
* updated docs and configs for d4rl v2 envs; added iql results
* update iql template json config to reflect v0.3 changes
* generate iql configs for d4rl
* fixed small bug with d4rl
output dirs
* update version numbers for d4rl docs
* adding r3m and mvp pretraining
* clean
* add doc for pretrain
* fix
* revert change to env robosuite
* minor rename
* update pretrained reps tutorial
* fix minor issues
* small update to iql config
* small docs fix
* add orbit reference
* more 2023 papers
* working version of color jitter net
* add gaussian noise randomizer
* fix bug in GaussianNoiseRandomizer
* randomizer visualization
* center cropping for CropRandomizer eval mode
* viz randomizer
* move randomizers to obs_core.py
* remove dups
* remove dups
* missing import
* doc instruction for visualizing randomizer
* Fix tab issue
* update reqs with matplotlib
* modified docs for new datasets, new algos, wandb support
* add note about using offline_study datasets
* postprocess xml for v1.3
* update postprocess function
* added verification exps
* limit num torch threads
* updated dataset gen script
* mj211 verification runs
* started conversion script for robosuite v1.3.1. WIP
* initial conversion script ready
* add verion to env_robosuite; fix bug from observation modality PR
* added v_1_3_1 verification jsons
* converted lift dataset, wip on can
* playback with can appears working but state divergence still high
* all tasks minimally in place now without mujoco exceptions
* minimal script for robosuite v140 conversion; created configs
* updated robosuite v140 convertion script
* added version checks for robosuite
* update dataset registry to point to v141 raw datasets
* generating and downloading datasets for v141
* updated obs extraction script to use v141 robosuite datasets
* remove unsued scripts
* remove unnec script
* delete old verification scripts
* remove unnec changes
* add low_dim dataset links back in, more informative message for unstored datasets
* Transformer support (#38)
* initial BC-mingpt implementation
* added adamw and weight scheduler
* updated mingpt configs for sweep
* rename mingpt -> transformer
* cleaned up bc transformer in bc.py
* cleaned up code comments
* remove internal config gen scripts
* addressing Danfei's comments in PR
* update template configs and make some cosmetic changes
* add ack for mingpt
---------
Co-authored-by: Ajay Mandlekar
Co-authored-by: amandlek
* bump version and minor docs update
* remove file
* some minor docs fixes for updated test_v141 hdf5
* small change to gitignore for test hdf5
* some minor docs fixes
* update 2023 refs
* Update obs_core.py with import
* Add wandb description in "viewing results" tutorial page
* added more wandb info to docs
* fix observation postprocessing for nested obs dicts
* added bc transformer tests
* V0.3 transformer tutorial (#59)
* added transformer tutorial
* Update bc_transformer.json
* Update bc_transformer.json
* Update training_transformers.md
* adjusting tutorial docs based on feedback
* changes to pretrained representations and transformer tutorial
* added data loader params for pretrained reps tutorial
---------
Co-authored-by: Bronars
* add readme note
* V0.3 pretrained rep fix (#64)
* fixed error in base_nets for R3MConv
* added error messages for Sequential
* added doc string for Sequential init
---------
Co-authored-by: Danfei Xu
Co-authored-by: snasiriany
Co-authored-by: josiah_wong <84cremebrule@gmail.com>
Co-authored-by: Soroush Nasiriany
Co-authored-by: Soroush Nasiriany
Co-authored-by: j96w
Co-authored-by: danfei
Co-authored-by: MBronars <35738375+MBronars@users.noreply.github.com>
Co-authored-by: Bronars
---
.gitignore | 4 +
README.md | 11 +-
docs/api/robomimic.algo.rst | 8 +
docs/api/robomimic.config.rst | 8 +
docs/api/robomimic.envs.rst | 8 +
docs/api/robomimic.models.rst | 16 +
docs/api/robomimic.rst | 11 +
docs/api/robomimic.utils.rst | 8 -
docs/conf.py | 9 +-
docs/datasets/d4rl.md | 30 +-
docs/datasets/overview.md | 6 +-
docs/datasets/robomimic_v0.1.md | 43 +-
docs/datasets/robosuite.md | 16 +-
docs/images/core_features.png | Bin 545237 -> 495907 bytes
docs/index.rst | 7 +
docs/introduction/getting_started.md | 12 +-
docs/introduction/implemented_algorithms.md | 8 +
docs/introduction/installation.md | 17 +-
docs/introduction/overview.md | 3 +-
docs/miscellaneous/acknowledgments.md | 1 +
docs/miscellaneous/contributing.md | 1 +
docs/miscellaneous/references.md | 29 +
docs/miscellaneous/team.md | 5 +-
docs/miscellaneous/troubleshooting.md | 2 +-
docs/model_zoo/robomimic_v0.1.md | 2 +-
docs/modules/dataset.md | 10 +-
docs/modules/models.md | 17 +-
docs/robomimic_logo.png | Bin 0 -> 20778 bytes
docs/tutorials/configs.md | 4 +-
docs/tutorials/dataset_contents.md | 14 +-
docs/tutorials/hyperparam_scan.md | 26 +-
docs/tutorials/pretrained_representations.md | 42 +
docs/tutorials/training_transformers.md | 43 +
docs/tutorials/viewing_results.md | 14 +-
docs/versions/v0.2.md | 18 +
examples/simple_obs_nets.py | 5 +-
examples/simple_train_loop.py | 1 +
examples/train_bc_rnn.py | 22 +-
requirements.txt | 1 +
robomimic/__init__.py | 33 +-
robomimic/algo/__init__.py | 1 +
robomimic/algo/algo.py | 42 +
robomimic/algo/bc.py | 255 +++++-
robomimic/algo/bcq.py | 4 +-
robomimic/algo/cql.py | 4 +-
robomimic/algo/gl.py | 8 +-
robomimic/algo/hbc.py | 6 +-
robomimic/algo/iql.py | 428 +++++++++
robomimic/algo/iris.py | 6 +-
robomimic/algo/td3_bc.py | 4 +-
robomimic/config/__init__.py | 1 +
robomimic/config/base_config.py | 63 +-
robomimic/config/bc_config.py | 39 +-
robomimic/config/bcq_config.py | 1 +
.../default_templates/bc_transformer.json | 171 ++++
robomimic/config/iql_config.py | 73 ++
robomimic/envs/env_base.py | 8 +
robomimic/envs/env_robosuite.py | 26 +-
robomimic/envs/wrappers.py | 220 +++++
robomimic/exps/templates/bc.json | 122 +--
robomimic/exps/templates/bcq.json | 104 +--
robomimic/exps/templates/cql.json | 104 +--
robomimic/exps/templates/gl.json | 104 +--
robomimic/exps/templates/hbc.json | 210 +----
robomimic/exps/templates/iql.json | 192 ++++
robomimic/exps/templates/iris.json | 298 ++-----
robomimic/exps/templates/td3_bc.json | 102 +--
robomimic/macros.py | 27 +
robomimic/models/__init__.py | 2 +-
robomimic/models/base_nets.py | 708 +++++----------
robomimic/models/obs_core.py | 828 ++++++++++++++++++
robomimic/models/obs_nets.py | 248 +++++-
robomimic/models/policy_nets.py | 361 +++++++-
robomimic/models/transformers.py | 426 +++++++++
robomimic/scripts/conversion/convert_d4rl.py | 4 +-
.../scripts/conversion/convert_robosuite.py | 1 +
robomimic/scripts/dataset_states_to_obs.py | 31 +-
robomimic/scripts/download_datasets.py | 9 +-
.../scripts/extract_obs_from_raw_datasets.sh | 112 +--
robomimic/scripts/generate_paper_configs.py | 80 +-
robomimic/scripts/setup_macros.py | 32 +
robomimic/scripts/train.py | 26 +-
robomimic/utils/dataset.py | 18 +-
robomimic/utils/env_utils.py | 51 +-
robomimic/utils/file_utils.py | 157 ++--
robomimic/utils/hyperparam_utils.py | 53 +-
robomimic/utils/log_utils.py | 100 ++-
robomimic/utils/macros.py | 6 -
robomimic/utils/obs_utils.py | 2 +-
robomimic/utils/python_utils.py | 2 +-
robomimic/utils/tensor_utils.py | 4 +-
robomimic/utils/test_utils.py | 8 +-
robomimic/utils/torch_utils.py | 50 +-
robomimic/utils/train_utils.py | 44 +-
robomimic/utils/vis_utils.py | 38 +
setup.py | 5 +-
tests/test_bc.py | 17 +
tests/test_iql.py | 143 +++
98 files changed, 4997 insertions(+), 1707 deletions(-)
create mode 100644 docs/robomimic_logo.png
create mode 100644 docs/tutorials/pretrained_representations.md
create mode 100644 docs/tutorials/training_transformers.md
create mode 100644 docs/versions/v0.2.md
create mode 100644 robomimic/algo/iql.py
create mode 100644 robomimic/config/default_templates/bc_transformer.json
create mode 100644 robomimic/config/iql_config.py
create mode 100644 robomimic/envs/wrappers.py
create mode 100644 robomimic/exps/templates/iql.json
create mode 100644 robomimic/macros.py
create mode 100644 robomimic/models/obs_core.py
create mode 100644 robomimic/models/transformers.py
create mode 100644 robomimic/scripts/setup_macros.py
delete mode 100644 robomimic/utils/macros.py
create mode 100644 tests/test_iql.py
diff --git a/.gitignore b/.gitignore
index 65249c39..d53318df 100644
--- a/.gitignore
+++ b/.gitignore
@@ -6,6 +6,7 @@ dist/
# local test dataset that is lazily downloaded by example scripts
tests/assets/test.hdf5
+tests/assets/test_v141.hdf5
# pycharm configs
.idea/
@@ -119,3 +120,6 @@ venv.bak/
.mypy_cache/
*.mp4
*.pth
+
+# private macros
+macros_private.py
diff --git a/README.md b/README.md
index 28a5dd27..cdd6cf4a 100644
--- a/README.md
+++ b/README.md
@@ -15,10 +15,19 @@
-------
## Latest Updates
+- [07/03/2023] **v0.3**: BC-Transformer and IQL :brain:, support for DeepMind MuJoCo bindings :robot:, pre-trained image reps :eye:, wandb logging :chart_with_upwards_trend:, and more
- [05/23/2022] **v0.2.1**: Updated website and documentation to feature more tutorials :notebook_with_decorative_cover:
-- [12/16/2021] **v0.2.0**: Modular observation modalities and encoders :wrench:, support for [MOMART](https://sites.google.com/view/il-for-mm/home) datasets :open_file_folder:
+- [12/16/2021] **v0.2.0**: Modular observation modalities and encoders :wrench:, support for [MOMART](https://sites.google.com/view/il-for-mm/home) datasets :open_file_folder: [[release notes]](https://github.com/ARISE-Initiative/robomimic/releases/tag/v0.2.0) [[documentation]](https://robomimic.github.io/docs/v0.2/introduction/overview.html)
- [08/09/2021] **v0.1.0**: Initial code and paper release
+-------
+
+## Colab quickstart
+Get started with a quick colab notebook demo of robomimic with installing anything locally.
+
+[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1b62r_km9pP40fKF0cBdpdTO2P_2eIbC6?usp=sharing)
+
+
-------
**robomimic** is a framework for robot learning from demonstration.
diff --git a/docs/api/robomimic.algo.rst b/docs/api/robomimic.algo.rst
index 7a6dd1ed..c2b32890 100644
--- a/docs/api/robomimic.algo.rst
+++ b/docs/api/robomimic.algo.rst
@@ -52,6 +52,14 @@ robomimic.algo.hbc module
:undoc-members:
:show-inheritance:
+robomimic.algo.iql module
+-------------------------
+
+.. automodule:: robomimic.algo.iql
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
robomimic.algo.iris module
--------------------------
diff --git a/docs/api/robomimic.config.rst b/docs/api/robomimic.config.rst
index e7a4b32b..1c53be6c 100644
--- a/docs/api/robomimic.config.rst
+++ b/docs/api/robomimic.config.rst
@@ -60,6 +60,14 @@ robomimic.config.hbc\_config module
:undoc-members:
:show-inheritance:
+robomimic.config.iql\_config module
+-----------------------------------
+
+.. automodule:: robomimic.config.iql_config
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
robomimic.config.iris\_config module
------------------------------------
diff --git a/docs/api/robomimic.envs.rst b/docs/api/robomimic.envs.rst
index 94bfb690..5e7ab732 100644
--- a/docs/api/robomimic.envs.rst
+++ b/docs/api/robomimic.envs.rst
@@ -36,6 +36,14 @@ robomimic.envs.env\_robosuite module
:undoc-members:
:show-inheritance:
+robomimic.envs.wrappers module
+------------------------------
+
+.. automodule:: robomimic.envs.wrappers
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
Module contents
---------------
diff --git a/docs/api/robomimic.models.rst b/docs/api/robomimic.models.rst
index a81e10a9..a50f8760 100644
--- a/docs/api/robomimic.models.rst
+++ b/docs/api/robomimic.models.rst
@@ -20,6 +20,14 @@ robomimic.models.distributions module
:undoc-members:
:show-inheritance:
+robomimic.models.obs\_core module
+---------------------------------
+
+.. automodule:: robomimic.models.obs_core
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
robomimic.models.obs\_nets module
---------------------------------
@@ -36,6 +44,14 @@ robomimic.models.policy\_nets module
:undoc-members:
:show-inheritance:
+robomimic.models.transformers module
+------------------------------------
+
+.. automodule:: robomimic.models.transformers
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
robomimic.models.vae\_nets module
---------------------------------
diff --git a/docs/api/robomimic.rst b/docs/api/robomimic.rst
index e40292b2..0d8ad0a7 100644
--- a/docs/api/robomimic.rst
+++ b/docs/api/robomimic.rst
@@ -13,6 +13,17 @@ Subpackages
robomimic.models
robomimic.utils
+Submodules
+----------
+
+robomimic.macros module
+-----------------------
+
+.. automodule:: robomimic.macros
+ :members:
+ :undoc-members:
+ :show-inheritance:
+
Module contents
---------------
diff --git a/docs/api/robomimic.utils.rst b/docs/api/robomimic.utils.rst
index 57160ef4..51095559 100644
--- a/docs/api/robomimic.utils.rst
+++ b/docs/api/robomimic.utils.rst
@@ -52,14 +52,6 @@ robomimic.utils.loss\_utils module
:undoc-members:
:show-inheritance:
-robomimic.utils.macros module
------------------------------
-
-.. automodule:: robomimic.utils.macros
- :members:
- :undoc-members:
- :show-inheritance:
-
robomimic.utils.obs\_utils module
---------------------------------
diff --git a/docs/conf.py b/docs/conf.py
index 7402a994..59eff968 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -59,17 +59,17 @@
# General information about the project.
project = 'robomimic'
-copyright = '2022, Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang'
-author = 'Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang'
+copyright = 'the robomimic core team, 2023'
+author = 'the robomimic core team'
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
-version = robomimic.__version__
+version = (".").join(robomimic.__version__.split(".")[:-1])
# The full version, including alpha/beta/rc tags.
-release = robomimic.__version__
+release = (".").join(robomimic.__version__.split(".")[:-1])
# The language for content autogenerated by Sphinx. Refer to documentation
@@ -98,6 +98,7 @@
# a list of builtin themes.
#
html_theme = 'sphinx_book_theme'
+html_logo = "robomimic_logo.png"
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
diff --git a/docs/datasets/d4rl.md b/docs/datasets/d4rl.md
index ba19e82a..9e7165f5 100644
--- a/docs/datasets/d4rl.md
+++ b/docs/datasets/d4rl.md
@@ -9,9 +9,9 @@ Use `convert_d4rl.py` in the `scripts/conversion` folder to automatically downlo
```sh
# by default, download to robomimic/datasets
-$ python convert_d4rl.py --env walker2d-medium-expert-v0
+$ python convert_d4rl.py --env walker2d-medium-expert-v2
# download to specific folder
-$ python convert_d4rl.py --env walker2d-medium-expert-v0 --folder /path/to/output/folder/
+$ python convert_d4rl.py --env walker2d-medium-expert-v2 --folder /path/to/output/folder/
```
- `--env` specifies the dataset to download
@@ -25,19 +25,19 @@ No postprocessing is required, assuming the above script is run!
## D4RL Results
-Below, we provide a table of results on common D4RL datasets using the algorithms included in the released codebase. We follow the convention in the TD3-BC paper, where we average results over the final 10 rollout evaluations, but we use 50 rollouts instead of 10 for each evaluation. Apart from a small handful of the halfcheetah results, the results align with those presented in the [TD3_BC paper](https://arxiv.org/abs/2106.06860). We suspect the halfcheetah results are different because we used `mujoco-py` version `2.0.2.13` in our evaluations, as opposed to `1.5` in order to be consistent with the version we were using for robosuite datasets. The results below were generated with `gym` version `0.17.3` and this `d4rl` [commit](https://github.com/rail-berkeley/d4rl/tree/9b68f31bab6a8546edfb28ff0bd9d5916c62fd1f).
+Below, we provide a table of results on common D4RL datasets using the algorithms included in the released codebase. We follow the convention in the TD3-BC paper, where we average results over the final 10 rollout evaluations, but we use 50 rollouts instead of 10 for each evaluation. All results are reported on the `-v2` environment variants. Apart from a small handful of the halfcheetah results, the results align with those presented in the [TD3_BC paper](https://arxiv.org/abs/2106.06860). We suspect the halfcheetah results are different because we used `mujoco-py` version `2.1.2.14` in our evaluations, as opposed to `1.5` in order to be consistent with the version we were using for robosuite datasets. The results below were generated with `gym` version `0.24.1` and this `d4rl` [commit](https://github.com/Farama-Foundation/D4RL/tree/305676ebb2e26582d50c6518c8df39fd52dea587).
-| | **BCQ** | **CQL** | **TD3-BC** |
-| ----------------------------- | ------------- | ------------- | ------------- |
-| **HalfCheetah-Medium** | 40.8% (4791) | 38.5% (4497) | 41.7% (4902) |
-| **Hopper-Medium** | 36.9% (1181) | 30.7% (980) | 97.9% (3167) |
-| **Walker2d-Medium** | 66.4% (3050) | 65.2% (2996) | 77.0% (3537) |
-| **HalfCheetah-Medium-Expert** | 74.9% (9016) | 21.5% (2389) | 79.4% (9578) |
-| **Hopper-Medium-Expert** | 83.8% (2708) | 111.7% (3614) | 112.2% (3631) |
-| **Walker2d-Medium-Expert** | 70.2% (3224) | 77.4% (3554) | 102.0% (4683) |
-| **HalfCheetah-Expert** | 94.3% (11427) | 29.2% (3342) | 95.4% (11569) |
-| **Hopper-Expert** | 104.7% (3389) | 111.8% (3619) | 112.2% (3633) |
-| **Walker2d-Expert** | 80.5% (3699) | 108.0% (4958) | 105.3% (4837) |
+| | **BCQ** | **CQL** | **TD3-BC** | **IQL** |
+| ----------------------------- | ------------- | ------------- | ------------- | ------------- |
+| **HalfCheetah-Medium** | 46.8% (5535) | 46.7% (5516) | 47.9% (5664) | 45.6% (5379) |
+| **Hopper-Medium** | 63.9% (2059) | 59.2% (1908) | 61.0% (1965) | 53.7% (1729) |
+| **Walker2d-Medium** | 74.6% (3426) | 79.7% (3659) | 82.9% (3806) | 77.0% (3537) |
+| **HalfCheetah-Medium-Expert** | 89.9% (10875) | 77.6% (9358) | 92.1% (11154) | 89.0% (10773) |
+| **Hopper-Medium-Expert** | 79.5% (2566) | 62.9% (2027) | 89.7% (2900) | 110.1% (3564) |
+| **Walker2d-Medium-Expert** | 98.7% (4535) | 109.0% (5007) | 111.1% (5103) | 109.7% (5037) |
+| **HalfCheetah-Expert** | 92.9% (11249) | 67.7% (8126) | 94.6% (11469) | 93.3% (11304) |
+| **Hopper-Expert** | 92.3% (2984) | 104.2% (3370) | 108.5% (3512) | 110.5% (3577) |
+| **Walker2d-Expert** | 108.6% (4987) | 108.5% (4983) | 110.3% (5066) | 109.1% (5008) |
### Reproducing D4RL Results
@@ -52,4 +52,4 @@ In order to reproduce the results above, first make sure that the `generate_pape
journal={arXiv preprint arXiv:2004.07219},
year={2020}
}
-```
\ No newline at end of file
+```
diff --git a/docs/datasets/overview.md b/docs/datasets/overview.md
index ac9731e7..0f260e59 100644
--- a/docs/datasets/overview.md
+++ b/docs/datasets/overview.md
@@ -159,8 +159,6 @@ You can easily list the filter keys present in a dataset with the `get_dataset_i
-Using filter keys during training is easy. To use the generated train-valid split, you can set `config.experiment.validate=True` so that the demos under `mask/train` are used for training, and the demos under `mask/valid` are used for validation.
-
-You can also use a custom filter key for training by setting `config.train.hdf5_filter_key=`. This ensures that only the demos under `mask/` are used during training. If you also set `config.experiment.validate=True`, this filter key's train-valid split will be used.
-
+Using filter keys during training is easy. To use the generated train-valid split, you can set `config.experiment.validate=True` to ensure that validation will run after each training epoch, and then set `config.train.hdf5_filter_key="train"` and `config.train.hdf5_validation_filter_key="valid"` so that the demos under `mask/train` are used for training, and the demos under `mask/valid` are used for validation.
+You can also use a custom filter key for training by setting `config.train.hdf5_filter_key=`. This ensures that only the demos under `mask/` are used during training. You can also specify a custom filter key for validation by setting `config.train.hdf5_validation_filter_key`.
diff --git a/docs/datasets/robomimic_v0.1.md b/docs/datasets/robomimic_v0.1.md
index d8134ec9..43e2b047 100644
--- a/docs/datasets/robomimic_v0.1.md
+++ b/docs/datasets/robomimic_v0.1.md
@@ -12,8 +12,11 @@ robomimic v0.1 datasets is a large-scale, diverse collection of task demonstrati
Warning!
+
-When working with these datasets, please make sure that you have installed [robosuite](https://robosuite.ai/) from source and are on the `offline_study` branch.
+Since the release of robomimic v0.3, we recommend that users install [robosuite](https://robosuite.ai/) from source and are on the `v1.4.1` branch. The `v1.4.1` branch of robosuite uses Google DeepMind's [mujoco python bindings](https://github.com/deepmind/mujoco) which replaces the now deprecated [mujoco-py](https://github.com/openai/mujoco-py).
+
+Note that the CoRL 2021 datasets relied on the mujoco-py based `offline_study` branch of robosuite. In this version of robomimic we provide users with datasets based on `v1.4.1`, which we found produce similar results to the original `offline_study` datasets. However if you would like to download the `offline_study` datasets, please use the `v0.2.0` branch of robomimic and consult the [v0.2 docs](https://robomimic.github.io/docs/v0.2/datasets/robomimic_v0.1.html) for instructions.
@@ -33,8 +36,8 @@ $ python download_datasets.py
$ python download_datasets.py --tasks sim --dataset_types ph --hdf5_types low_dim --dry_run
$ python download_datasets.py --tasks sim --dataset_types ph --hdf5_types low_dim
-# download all low-dim and image multi-human datasets for the can and square tasks
-$ python download_datasets.py --tasks can square --dataset_types mh --hdf5_types low_dim image
+# download all low-dim multi-human datasets for the can and square tasks
+$ python download_datasets.py --tasks can square --dataset_types mh --hdf5_types low_dim
# download the sparse reward machine-generated low-dim datasets
$ python download_datasets.py --tasks all --dataset_types mg --hdf5_types low_dim_sparse
@@ -61,9 +64,8 @@ We also provide direct download links for each hdf5 dataset (the download links
| **Lift
(PH)** | **Can
(PH)** | **Square
(PH)** | **Transport
(PH)** | **Tool Hang
(PH)** |
| :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
| ![lift](../images/lift.png) | ![can](../images/can.png) | ![square](../images/square.png) | ![transport](../images/transport.png) | ![tool_hang](../images/tool_hang.png) |
-| [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/ph/demo.hdf5)
(21 MB) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/ph/demo.hdf5)
(45 MB) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/square/ph/demo.hdf5)
(49 MB) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/transport/ph/demo.hdf5)
(185 MB) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/tool_hang/ph/demo.hdf5)
(127 MB) |
-| [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/ph/low_dim.hdf5)
(18 MB) | [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/ph/low_dim.hdf5)
(44 MB) | [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/square/ph/low_dim.hdf5)
(48 MB) | [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/transport/ph/low_dim.hdf5)
(296 MB) | [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/tool_hang/ph/low_dim.hdf5)
(193 MB) |
-| [image](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/ph/image.hdf5)
(801 MB) | [image](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/ph/image.hdf5)
(1.9 GB) | [image](http://downloads.cs.stanford.edu/downloads/rt_benchmark/square/ph/image.hdf5)
(2.5 GB) | [image](http://downloads.cs.stanford.edu/downloads/rt_benchmark/transport/ph/image.hdf5)
(16 GB) | [image](http://downloads.cs.stanford.edu/downloads/rt_benchmark/tool_hang/ph/image.hdf5)
(63 GB) |
+| [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/ph/demo_v141.hdf5) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/ph/demo_v141.hdf5) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/square/ph/demo_v141.hdf5) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/transport/ph/demo_v141.hdf5) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/tool_hang/ph/demo_v141.hdf5) |
+| [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/ph/low_dim_v141.hdf5) | [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/ph/low_dim_v141.hdf5) | [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/square/ph/low_dim_v141.hdf5) | [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/transport/ph/low_dim_v141.hdf5) | [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/tool_hang/ph/low_dim_v141.hdf5) |
| **Lift Real
(PH)** | **Can Real
(PH)** | **Tool Hang Real
(PH)** |
| :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
@@ -86,10 +88,8 @@ We also provide direct download links for each hdf5 dataset (the download links
| **Lift
(MH)** | **Can
(MH)** | **Square
(MH)** | **Transport
(MH)** |
| :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: | :----------------------------------------------------------: |
| ![lift](../images/lift.png) | ![can](../images/can.png) | ![square](../images/square.png) | ![transport](../images/transport.png) |
-| [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/mh/demo.hdf5)
(20 MB) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/mh/demo.hdf5)
(51 MB) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/square/mh/demo.hdf5)
(45 MB) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/transport/mh/demo.hdf5)
(212 MB) |
-| [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/mh/low_dim.hdf5)
(46 MB) | [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/mh/low_dim.hdf5)
(108 MB) | [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/square/mh/low_dim.hdf5)
(119 MB) | [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/transport/mh/low_dim.hdf5)
(609 MB) |
-| [image](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/mh/image.hdf5)
(2.6 GB) | [image](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/mh/image.hdf5)
(5.1 GB) | [image](http://downloads.cs.stanford.edu/downloads/rt_benchmark/square/mh/image.hdf5)
(6.5 GB) | [image](http://downloads.cs.stanford.edu/downloads/rt_benchmark/transport/mh/image.hdf5)
(32 GB) |
-
+| [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/mh/demo_v141.hdf5) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/mh/demo_v141.hdf5) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/square/mh/demo_v141.hdf5) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/transport/mh/demo_v141.hdf5) |
+| [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/mh/low_dim_v141.hdf5) | [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/mh/low_dim_v141.hdf5) | [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/square/mh/low_dim_v141.hdf5) | [low_dim](http://downloads.cs.stanford.edu/downloads/rt_benchmark/transport/mh/low_dim_v141.hdf5) |
@@ -107,11 +107,9 @@ We also provide direct download links for each hdf5 dataset (the download links
| **Lift
(MG)** | **Can
(MG)** |
| :----------------------------------------------------------: | :----------------------------------------------------------: |
| ![lift](../images/lift.png) | ![can](../images/can.png) |
-| [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/mg/demo.hdf5)
(96 MB) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/mg/demo.hdf5)
(457 MB) |
-| [low_dim (sparse)](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/mg/low_dim_sparse.hdf5)
(303 MB) | [low_dim (sparse)](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/mg/low_dim_sparse.hdf5)
(1.1 GB) |
-| [low_dim (dense)](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/mg/low_dim_dense.hdf5)
(303 MB) | [low_dim (dense)](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/mg/low_dim_dense.hdf5)
(1.1 GB) |
-| [image (sparse)](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/mg/image_sparse.hdf5)
(19 GB) | [image (sparse)](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/mg/image_sparse.hdf5)
(48 GB) |
-| [image (dense)](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/mg/image_dense.hdf5)
(19 GB) | [image (dense)](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/mg/image_dense.hdf5)
(48 GB) |
+| [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/mg/demo_v141.hdf5) | [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/mg/demo_v141.hdf5) |
+| [low_dim (sparse)](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/mg/low_dim_sparse_v141.hdf5) | [low_dim (sparse)](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/mg/low_dim_sparse_v141.hdf5) |
+| [low_dim (dense)](http://downloads.cs.stanford.edu/downloads/rt_benchmark/lift/mg/low_dim_dense_v141.hdf5) | [low_dim (dense)](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/mg/low_dim_dense_v141.hdf5) |
@@ -128,9 +126,8 @@ We also provide direct download links for each hdf5 dataset (the download links
| **Can Paired** |
| :----------------------------------------------------------: |
| |
-| [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/paired/demo.hdf5)
(39 MB) |
-| [low_dim (sparse)](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/paired/low_dim.hdf5)
(39 MB) |
-| [image (sparse)](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/paired/image.hdf5)
(1.7 GB) |
+| [raw](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/paired/demo_v141.hdf5) |
+| [low_dim (sparse)](http://downloads.cs.stanford.edu/downloads/rt_benchmark/can/paired/low_dim_v141.hdf5) |
@@ -138,7 +135,7 @@ We also provide direct download links for each hdf5 dataset (the download links
## Postprocessing
If a **low_dim** or **image** dataset was downloaded, the dataset works out of the box! No postprocessing is needed.
-If a **raw** dataset was downloaded, the dataset must be postprocessed since there are no observations stored. You must run `dataset_states_to_obs.py`. For more information, see [this page](robosuite.html#extracting-observations-from-mujoco-states).
+If a **raw** dataset was downloaded, the dataset must be postprocessed since there are no observations stored. To extract observations, please see the `extract_obs_from_raw_datasets.sh` script.
## Info
@@ -191,7 +188,7 @@ python /path/to/robomimic/scripts/train.py --config /path/to/robomimic/exps/pape