Revamp docs fixes (#31)

* initial refactor of docs * add placeholder for reproducing experiments tutorial page * updated draft of introduction/overview.md * updated getting started page * getting_started.md: refactor #2 * small change * small change * html boxes for core features * changing some wording * small changes * adding datasets ipython notebook * testing new warning dropdown * update style * run policy tutorial notebook * importing roboturk system features content; wip * added some warning blocks and reference in docs for ipython dataset notebook * update intro WIP * add backwards compatibility functionality for loading configs from checkpoints * finish update of installation doc * better feature box * run policy notebook works now * shortened overview * update the datasets pages * update theme and table * added core features diagram * update core features * updated getting started page * fixed warning box in getting_started * fix folder for core_features.png img * update dataset page * update year in sphinx conf * add train-val split info * notebook and updated doc for pretrained model tutorial * update core_features image; update links to next steps in getting_started page * updated modules overview * fix path to image in modules/overview.md * add link to creating custom env wrapper from dataset overview page * minor updates * fixed several broken links, and style changes * Add missing dependency in requirements-doc.txt * minor reordering in installation docs page * fix datasets doc * fix datasets doc * Fix dataset overview table not centered issue * Remove redundant period * make hdf5 structure dropdown more prominent, remove unused tutorial * restructure a few tutorials with placeholders * fix numbering on installation page, trim down verbosity on robosuite dataset page, move info around * (docs) remove first sentence from contributing.md * (docs) update next links in getting_started.md * (docs) add note on --debug flag in getting_started.md * (docs) added implemented_algorithms.md page in intro section * (docs) initial work on configuring logging in viewing_results.md * (docs) added details on viewing training results in viewing_results.md * replace configs tutorial with ways to launch runs * update next steps and rename dataset tutorial * cleanup dataset contents tutorial and add reproduce experiments tutorial content * slight refactor of algo tutorial * (docs) refined algorithms.md and viewing_results.md * (docs) hyperparam_scan.md less verbose (wip) * fix missing links, modify warning * fix missing links, modify warning * reorder tutorials * (docs) updated logging and viewing results page - json comments, viewing tensorboard results * (docs) updated hyperparam_scan page * (docs) fix dataset sturcture * condense code blocks and make obs modalities into bullet points * Warning blocks in config.md * bump version, add tutorial links * try to fix readme gifs * try one more time * put core features image back in readme * fix readme links * remove some prints Co-authored-by: snasiriany <[email protected]> Co-authored-by: josiah_wong <[email protected]> Co-authored-by: Danfei Xu <[email protected]> Co-authored-by: j96w <[email protected]> Co-authored-by: Yifeng Zhu <[email protected]>
ARISE-Initiative · May 24, 2022 · 5f328b9 · 5f328b9
1 parent 4a39b1d
commit 5f328b9
Show file tree

Hide file tree

Showing 42 changed files with 3,516 additions and 1,124 deletions.
diff --git a/.gitignore b/.gitignore
@@ -1,8 +1,8 @@
 # pip distribution folder
 dist/
 
-# datasets folder
-datasets/
+# datasets folder at top-level (leading slash)
+/datasets/
 
 # local test dataset that is lazily downloaded by example scripts
 tests/assets/test.hdf5

diff --git a/README.md b/README.md
@@ -11,53 +11,70 @@
   <img width="24.0%" src="docs/images/task_transport.gif">
  </p>
 
-[**[Homepage]**](https://arise-initiative.github.io/robomimic-web/) &ensp; [**[Documentation]**](https://arise-initiative.github.io/robomimic-web/docs/introduction/overview.html) &ensp; [**[Study Paper]**](https://arxiv.org/abs/2108.03298) &ensp; [**[Study Website]**](https://arise-initiative.github.io/robomimic-web/study/) &ensp; [**[ARISE Initiative]**](https://github.com/ARISE-Initiative)
+[**[Homepage]**](https://robomimic.github.io/) &ensp; [**[Documentation]**](https://robomimic.github.io/docs/introduction/overview.html) &ensp; [**[Study Paper]**](https://arxiv.org/abs/2108.03298) &ensp; [**[Study Website]**](https://robomimic.github.io/study/) &ensp; [**[ARISE Initiative]**](https://github.com/ARISE-Initiative)
 
 -------
 ## Latest Updates
+- [05/23/2022] **v0.2.1**: Updated website and documentation to feature more tutorials :notebook_with_decorative_cover:
 - [12/16/2021] **v0.2.0**: Modular observation modalities and encoders :wrench:, support for [MOMART](https://sites.google.com/view/il-for-mm/home) datasets :open_file_folder:
 - [08/09/2021] **v0.1.0**: Initial code and paper release
 
 -------
 
-**robomimic** is a framework for robot learning from demonstration. It offers a broad set of demonstration datasets collected on robot manipulation domains, and learning algorithms to learn from these datasets. This project is part of the broader [Advancing Robot Intelligence through Simulated Environments (ARISE) Initiative](https://github.com/ARISE-Initiative), with the aim of lowering the barriers of entry for cutting-edge research at the intersection of AI and Robotics.
+**robomimic** is a framework for robot learning from demonstration.
+It offers a broad set of demonstration datasets collected on robot manipulation domains and offline learning algorithms to learn from these datasets.
+**robomimic** aims to make robot learning broadly *accessible* and *reproducible*, allowing researchers and practitioners to benchmark tasks and algorithms fairly and to develop the next generation of robot learning algorithms.
 
-Imitating human demonstrations is a promising approach to endow robots with various manipulation capabilities. While recent advances have been made in imitation learning and batch (offline) reinforcement learning, a lack of open-source human datasets and reproducible learning methods make assessing the state of the field difficult. The overarching goal of **robomimic** is to provide researchers and practitioners with:
+## Core Features
 
-- a **standardized set of large demonstration datasets** across several benchmarking tasks to facilitate fair comparisons, with a focus on learning from human-provided demonstrations
-- a **standardized set of large demonstration datasets** across several benchmarking tasks to facilitate fair comparisons, with a focus on learning from human-provided demonstrations (see [this link](https://arise-initiative.github.io/robomimic-web/docs/introduction/quickstart.html#supported-datasets) for a list of supported datasets)
-- **high-quality implementations of several learning algorithms** for training closed-loop policies from offline datasets to make reproducing results easy and lower the barrier to entry
-- a **modular design** that offers great flexibility in extending algorithms and designing new algorithms
+<p align="center">
+  <img width="50.0%" src="docs/images/core_features.png">
+ </p>
 
-This release of **robomimic** contains seven offline learning [algorithms](https://arise-initiative.github.io/robomimic-web/docs/modules/algorithms.html) and standardized [datasets](https://arise-initiative.github.io/robomimic-web/docs/introduction/results.html) collected across five simulated and three real-world multi-stage manipulation tasks of varying complexity. We highlight some features below (for a more thorough list of features, see [this link](https://arise-initiative.github.io/robomimic-web/docs/introduction/quickstart.html#features-overview)):
+<!-- **Standardized Datasets**
+- Simulated and real-world tasks
+- Multiple environments and robots
+- Diverse human-collected and machine-generated datasets
 
-- **standardized datasets:** a set of datasets collected from different sources (single proficient human, multiple humans, and machine-generated) across several simulated and real-world tasks, along with a plug-and-play [Dataset class](https://arise-initiative.github.io/robomimic-web/docs/modules/datasets.html) to easily use the datasets outside of this project
-- **algorithm implementations:** several high-quality implementations of offline learning algorithms, including BC, BC-RNN, HBC, IRIS, BCQ, CQL, and TD3-BC
-- **multiple observation spaces:** support for learning both low-dimensional and visuomotor policies, with support for observation tensor dictionaries throughout the codebase, making it easy to specify different subsets of observations to train a policy. This includes a set of useful tensor utilities to work with nested dictionaries of torch Tensors and numpy arrays.
-- **visualization utilities:** utilities for visualizing demonstration data, playing back actions, visualizing trained policies, and collecting new datasets using trained policies
-- **train launching utilities:** utilities for easily running hyperparameter sweeps, enabled by a flexible [Config](https://arise-initiative.github.io/robomimic-web/docs/modules/configs.html) management system
+**Suite of Learning Algorithms**
+- Imitation Learning algorithms (BC, BC-RNN, HBC)
+- Offline RL algorithms (BCQ, CQL, IRIS, TD3-BC)
 
-## Contributing to robomimic
+**Modular Design**
+- Low-dim + Visuomotor policies
+- Diverse network architectures
+- Support for external datasets
 
-This framework originally began development in late 2018. Researchers in the [Stanford Vision and Learning Lab](http://svl.stanford.edu/) (SVL) used it as an internal tool for training policies from offline human demonstration datasets. Now it is actively maintained and used for robotics research projects across multiple labs. We welcome community contributions to this project. For details please check our [contributing guidelines](https://arise-initiative.github.io/robomimic-web/docs/miscellaneous/contributing.html).
+**Flexible Workflow**
+- Hyperparameter sweep tools
+- Dataset visualization tools
+- Generating new datasets -->
 
-## Troubleshooting
 
-Please see the [troubleshooting](https://arise-initiative.github.io/robomimic-web/docs/miscellaneous/troubleshooting.html) section for common fixes, or [submit an issue](https://github.com/ARISE-Initiative/robomimic/issues) on our github page.
+## Reproducing benchmarks
 
-## Reproducing study results
+The robomimic framework also makes reproducing the results from different benchmarks and datasets easy. See the [datasets page](https://robomimic.github.io/docs/datasets/overview.html) for more information on downloading datasets and reproducing experiments.
 
-The **robomimic** framework also makes reproducing the results from this [study](https://arise-initiative.github.io/robomimic-web/study) easy. See the [results documentation](https://arise-initiative.github.io/robomimic-web/docs/introduction/results.html) for more information.
+## Troubleshooting
+
+Please see the [troubleshooting](https://robomimic.github.io/docs/miscellaneous/troubleshooting.html) section for common fixes, or [submit an issue](https://github.com/ARISE-Initiative/robomimic/issues) on our github page.
+
+## Contributing to robomimic
+This project is part of the broader [Advancing Robot Intelligence through Simulated Environments (ARISE) Initiative](https://github.com/ARISE-Initiative), with the aim of lowering the barriers of entry for cutting-edge research at the intersection of AI and Robotics.
+The project originally began development in late 2018 by researchers in the [Stanford Vision and Learning Lab](http://svl.stanford.edu/) (SVL).
+Now it is actively maintained and used for robotics research projects across multiple labs.
+We welcome community contributions to this project.
+For details please check our [contributing guidelines](https://robomimic.github.io/docs/miscellaneous/contributing.html).
 
-## Citations
+## Citation
 
 Please cite [this paper](https://arxiv.org/abs/2108.03298) if you use this framework in your work:
 
 ```bibtex
 @inproceedings{robomimic2021,
   title={What Matters in Learning from Offline Human Demonstrations for Robot Manipulation},
   author={Ajay Mandlekar and Danfei Xu and Josiah Wong and Soroush Nasiriany and Chen Wang and Rohun Kulkarni and Li Fei-Fei and Silvio Savarese and Yuke Zhu and Roberto Mart\'{i}n-Mart\'{i}n},
-  booktitle={arXiv preprint arXiv:2108.03298},
+  booktitle={Conference on Robot Learning (CoRL)},
   year={2021}
 }
 ```
diff --git a/docs/conf.py b/docs/conf.py
@@ -14,7 +14,7 @@
 import sys
 sys.path.insert(0, os.path.abspath('.'))
 
-import sphinx_rtd_theme
+import sphinx_book_theme
 import robomimic
 
 
@@ -29,7 +29,6 @@
 # ones.
 extensions = [
     'sphinx.ext.napoleon',
-    'sphinx_rtd_theme',
     'sphinx_markdown_tables',
     'sphinx.ext.mathjax',
     'sphinx.ext.githubpages',
@@ -60,7 +59,7 @@
 
 # General information about the project.
 project = 'robomimic'
-copyright = '2021, Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang'
+copyright = '2022, Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang'
 author = 'Ajay Mandlekar, Danfei Xu, Josiah Wong, Soroush Nasiriany, Chen Wang'
 
 # The version info for the project you're documenting, acts as replacement for
@@ -98,7 +97,7 @@
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
-html_theme = 'sphinx_rtd_theme'
+html_theme = 'sphinx_book_theme'
 
 # Theme options are theme-specific and customize the look and feel of a theme
 # further.  For a list of options available for each theme, see the
@@ -111,11 +110,11 @@
 # so a file named "default.css" will overwrite the builtin "default.css".
 html_static_path = ['_static']
 
-html_context = {
-    'css_files': [
-        '_static/theme_overrides.css',  # override wide tables in RTD theme
-    ],
-}
+# html_context = {
+#     'css_files': [
+#         '_static/theme_overrides.css',  # override wide tables in RTD theme
+#     ],
+# }
 
 # -- Options for HTMLHelp output ------------------------------------------
 

diff --git a/docs/datasets/d4rl.md b/docs/datasets/d4rl.md
@@ -0,0 +1,55 @@
+# D4RL
+
+## Overview
+The [D4RL](https://arxiv.org/abs/2004.07219) benchmark provides a set of locomotion tasks and demonstration datasets.
+
+## Downloading
+
+Use `convert_d4rl.py` in the `scripts/conversion` folder to automatically download and postprocess the D4RL dataset in a single step. For example:
+
+```sh
+# by default, download to robomimic/datasets
+$ python convert_d4rl.py --env walker2d-medium-expert-v0
+# download to specific folder
+$ python convert_d4rl.py --env walker2d-medium-expert-v0 --folder /path/to/output/folder/
+```
+
+- `--env` specifies the dataset to download
+- `--folder` specifies where you want to download the dataset. If no folder is provided, the `datasets` folder at the top-level of the repository will be used.
+
+The script will download the raw hdf5 dataset to `--folder`, and the converted one that is compatible with this repository into the `converted` subfolder.
+
+## Postprocessing
+
+No postprocessing is required, assuming the above script is run!
+
+## D4RL Results
+
+Below, we provide a table of results on common D4RL datasets using the algorithms included in the released codebase. We follow the convention in the TD3-BC paper, where we average results over the final 10 rollout evaluations, but we use 50 rollouts instead of 10 for each evaluation. Apart from a small handful of the halfcheetah results, the results align with those presented in the [TD3_BC paper](https://arxiv.org/abs/2106.06860). We suspect the halfcheetah results are different because we used `mujoco-py` version `2.0.2.13` in our evaluations, as opposed to `1.5` in order to be consistent with the version we were using for robosuite datasets. The results below were generated with `gym` version `0.17.3` and this `d4rl` [commit](https://github.com/rail-berkeley/d4rl/tree/9b68f31bab6a8546edfb28ff0bd9d5916c62fd1f).
+
+|                               | **BCQ**       | **CQL**       | **TD3-BC**    |
+| ----------------------------- | ------------- | ------------- | ------------- |
+| **HalfCheetah-Medium**        | 40.8% (4791)  | 38.5% (4497)  | 41.7% (4902)  |
+| **Hopper-Medium**             | 36.9% (1181)  | 30.7% (980)   | 97.9% (3167)  |
+| **Walker2d-Medium**           | 66.4% (3050)  | 65.2% (2996)  | 77.0% (3537)  |
+| **HalfCheetah-Medium-Expert** | 74.9% (9016)  | 21.5% (2389)  | 79.4% (9578)  |
+| **Hopper-Medium-Expert**      | 83.8% (2708)  | 111.7% (3614) | 112.2% (3631) |
+| **Walker2d-Medium-Expert**    | 70.2% (3224)  | 77.4% (3554)  | 102.0% (4683) |
+| **HalfCheetah-Expert**        | 94.3% (11427) | 29.2% (3342)  | 95.4% (11569) |
+| **Hopper-Expert**             | 104.7% (3389) | 111.8% (3619) | 112.2% (3633) |
+| **Walker2d-Expert**           | 80.5% (3699)  | 108.0% (4958) | 105.3% (4837) |
+
+
+### Reproducing D4RL Results
+
+In order to reproduce the results above, first make sure that the `generate_paper_configs.py` script has been run, where the `--dataset_dir` argument is consistent with the folder where the D4RL datasets were downloaded using the `convert_d4rl.py` script. This is also the first step for reproducing results on the released robot manipulation datasets. The `--config_dir` directory used in the script (`robomimic/exps/paper` by default) will contain a `d4rl.sh` script, and a `d4rl` subdirectory that contains all the json configs. The table results above can be generated simply by running the training commands in the shell script.
+
+## Citation
+```sh
+@article{fu2020d4rl,
+  title={D4rl: Datasets for deep data-driven reinforcement learning},
+  author={Fu, Justin and Kumar, Aviral and Nachum, Ofir and Tucker, George and Levine, Sergey},
+  journal={arXiv preprint arXiv:2004.07219},
+  year={2020}
+}
+```
diff --git a/docs/datasets/momart.md b/docs/datasets/momart.md
@@ -0,0 +1,57 @@
+# MOMART Datasets and Experiments
+
+## Overview
+[Mobile Manipulation RoboTurk (MoMaRT)](https://sites.google.com/view/il-for-mm/home) datasets are a collection of demonstrations collected on 5 long-horizon robot mobile manipulation tasks in a realistic simulated kitchen.
+
+<p align="center">
+  <img width="19.0%" src="../images/momart_table_setup_from_dishwasher_overview.png">
+  <img width="19.0%" src="../images/momart_table_setup_from_dresser_overview.png">
+  <img width="19.0%" src="../images/momart_table_cleanup_to_dishwasher_overview.png">
+  <img width="19.0%" src="../images/momart_table_cleanup_to_sink_overview.png">
+  <img width="19.0%" src="../images/momart_unload_dishwasher_to_dresser_overview.png">
+  <img width="19.0%" src="../images/momart_bowl_in_sink.png">
+  <img width="19.0%" src="../images/momart_dump_trash.png">
+  <img width="19.0%" src="../images/momart_grab_bowl.png">
+  <img width="19.0%" src="../images/momart_open_dishwasher.png">
+  <img width="19.0%" src="../images/momart_open_dresser.png">
+ </p>
+
+## Downloading
+
+
+<div class="admonition warning">
+<p class="admonition-title">Warning!</p>
+
+When working with these datasets, please make sure that you have installed [iGibson](http://svl.stanford.edu/igibson/) from source and are on the `momart` branch. Exact steps for installing can be found [HERE](https://sites.google.com/view/il-for-mm/datasets#h.qw0vufk0hknk).
+
+</div>
+
+We provide two ways for downloading MOMART datasets:
+
+### Method 1: Using `download_momart_datasets.py` (Recommended)
+`download_momart_datasets.py` is a python script that provides a programmatic way of installing all datasets. This is the preferred method, because this script also sets up a directory structure for the datasets that works out of the box with examples for reproducing [MOMART paper's](https://arxiv.org/abs/2112.05251) results.
+
+```sh
+# Use --help flag to view download and <ARG> options
+python <ROBOMIMIC_DIR>/robomimic/scripts/download_momart_datasets.py <ARGS> 
+```
+
+### Method 2: Using Direct Download Links
+
+For each type of dataset, we also provide a direct download links that will download the raw HDF5 file [HERE](https://sites.google.com/view/il-for-mm/datasets#h.ko0ilbky4y5u).
+
+## Postprocessing
+
+No postprocessing is needed for these datasets!
+
+## Citation
+```sh
+@inproceedings{wong2022error,
+  title={Error-Aware Imitation Learning from Teleoperation Data for Mobile Manipulation},
+  author={Wong, Josiah and Tung, Albert and Kurenkov, Andrey and Mandlekar, Ajay and Fei-Fei, Li and Savarese, Silvio and Mart{\'\i}n-Mart{\'\i}n, Roberto},
+  booktitle={Conference on Robot Learning},
+  pages={1367--1378},
+  year={2022},
+  organization={PMLR}
+}
+```