diff --git a/docs/_data/bringup.csv b/docs/_data/bringup.csv index a20af17..8a5616f 100644 --- a/docs/_data/bringup.csv +++ b/docs/_data/bringup.csv @@ -9,7 +9,7 @@ leader_modes_left,the file path to the 'mode config' YAML file for the left lead leader_modes_right,the file path to the 'mode config' YAML file for the right leader arm.,LocalVar('FindPackageShare\(pkg= ``aloha``\) + 'config' + 'leader_modes_right.yaml''), follower_modes_left,the file path to the 'mode config' YAML file for the left follower arm.,LocalVar('FindPackageShare\(pkg= ``aloha``\) + 'config' + 'follower_modes_left.yaml''), follower_modes_right,the file path to the 'mode config' YAML file for the right follower arm.,LocalVar('FindPackageShare\(pkg= ``aloha``\) + 'config' + 'follower_modes_right.yaml''), -launch_leaders,"if ``true``, launches both the leader and follower arms; if ``false, just the followers are launched.",``true``,"``true``, ``false``" +launch_leaders,"if ``true``, launches both the leader and follower arms; if ``false``, just the followers are launched.",``true``,"``true``, ``false``" use_cameras,"if ``true``, launches the camera drivers.",``true``,"``true``, ``false``" cam_high_name,,``cam_high``, cam_low_name,,``cam_low``, diff --git a/docs/operation/hugging_face.rst b/docs/operation/hugging_face.rst index 1f444ec..1c323ff 100644 --- a/docs/operation/hugging_face.rst +++ b/docs/operation/hugging_face.rst @@ -13,8 +13,8 @@ If you don't already have an account, sign up for a new account on the `Hugging Creating a New Dataset Repository --------------------------------- -Web Interface -^^^^^^^^^^^^^ +Web Interface (Repository) +^^^^^^^^^^^^^^^^^^^^^^^^^^ #. Navigate to the `Hugging Face website `_. #. Log in to your account. @@ -41,8 +41,8 @@ Uploading Your Dataset You have two primary methods to upload datasets: through the web interface or using the Python API. -Web Interface -^^^^^^^^^^^^^ +Web Interface (Upload Dataset) +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ #. Navigate to your dataset repository on the Hugging Face website. #. Click on the "Files and versions" tab. diff --git a/docs/operation/training.rst b/docs/operation/training.rst index 46d7e99..445195d 100644 --- a/docs/operation/training.rst +++ b/docs/operation/training.rst @@ -1,51 +1,228 @@ -======================= -Training and Evaluation -======================= +================================== +Training and Evaluation Guide +================================== -Virtual Environment Setup -========================= +This guide covers the training and evaluation workflows for **ACT** (Aloha Stationary) and **ACT++** (Aloha Mobile). + +Setup and Environment +===================== -Effective containerization is important when it comes to running machine learning models as there can be conflicting dependencies. -You can either use a Virtual Environment or Conda. +You can use either **Virtual Environment (VENV)** or **Conda** for containerization based on your preference. +Follow the steps below to set up your environment: -Virtual Environment Installation and Setup -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Virtual Environment Setup +-------------------------- #. Install the virtual environment package: - .. code-block:: bash + .. code-block:: bash + + sudo apt-get install python3-venv + +#. Create and activate a virtual environment: + + .. code-block:: bash + + python3 -m venv ~/act + source ~/act/bin/activate + +Conda Setup +----------- + +#. Create and activate a Conda virtual environment: - $ sudo apt-get install python3-venv + .. code-block:: bash + + conda create -n aloha python=3.8.10 -y + conda activate aloha + +ACT Workflow (Aloha Stationary) +=============================== + +Install Dependencies (ACT) +-------------------------- + +Install the necessary dependencies inside your containerized environment: + +.. code-block:: bash + + $ pip install dm_control==1.0.14 + $ pip install einops + $ pip install h5py + $ pip install ipython + $ pip install matplotlib + $ pip install mujoco==2.3.7 + $ pip install opencv-python + $ pip install packaging + $ pip install pexpect + $ pip install pyquaternion + $ pip install pyyaml + $ pip install rospkg + $ pip install torch + $ pip install torchvision -#. Create a virtual environment: +Clone ACT Repository +-------------------- - .. code-block:: bash +.. code-block:: bash - $ python3 -m venv ~/act # Creates a venv "act" in the home directory, can be created anywhere + $ cd ~ + $ git clone https://github.com/Interbotix/act.git act_training_evaluation -#. Activate the virtual environment: +Build and Install ACT Models +---------------------------- - .. code-block:: bash +.. code-block:: bash + :emphasize-lines: 4 + + ├── act + │   ├── assets + │   ├── constants.py + │   ├── detr + │   ├── ee_sim_env.py + │   ├── imitate_episodes.py + │   ├── __init__.py + │   ├── policy.py + │   ├── record_sim_episodes.py + │   ├── scripted_policy.py + │   ├── sim_env.py + │   ├── utils.py + │   └── visualize_episodes.py + ├── COLCON_IGNORE + ├── conda_env.yaml + ├── LICENSE + └── README.md + +Navigate to the ``detr`` directory inside the repository and install the detr module which contains the model definitions using the below command: + +.. code-block:: bash + + $ cd /path/to/act/detr && pip install -e . + +Training (ACT) +-------------- + +To start the training, follow the steps below: + +#. Sanity Check: + + Ensure you have all the hdf5 episodes located in the correct folder after following the data collection steps :ref:`operation/data_collection:Task Creation`. + +#. Source ROS Environment: + + .. code-block:: bash + + $ source /opt/ros/humble/setup.bash + $ source interbotix_ws/install/setup.bash + +#. Activate Virtual Environment: + + .. code-block:: bash $ source act/bin/activate -Conda Setup -^^^^^^^^^^^ +#. Start Training + + .. code-block:: bash + + $ cd /path/to/act/repository/ + $ python3 imitate_episodes.py \ + --task_name aloha_stationary_dummy \ + --ckpt_dir \ + --policy_class ACT \ + --kl_weight 10 \ + --chunk_size 100 \ + --hidden_dim 512 \ + --batch_size 8 \ + --dim_feedforward 3200 \ + --num_epochs 2000 \ + --lr 1e-5 \ + --seed 0 + +.. note:: + + - ``task_name`` argument should match one of the task names in the ``TASK_CONFIGS``, as configured in the :ref:`operation/data_collection:Task Creation` section. + - ``ckpt_dir``: The relative location where the checkpoints and best policy will be stored. + - ``policy_class``: Determines the choice of policy 'ACT'/'CNNMLP'. + - ``kl_weight``: Controls the balance between exploration and exploitation. + - ``chunk_size``: Determines the length of the action sequence. K=1 is no action chunking and K=episode length is full open loop control. + - ``batch_size``: Low batch size leads to better generalization and high batch size results in slower convergence but faster training time. + - ``num_epochs``: Too many epochs lead to overfitting; too few epochs may not allow the model to learn. + - ``lr``: Higher learning rate can lead to faster convergence but may overshoot the optima, while lower learning rate might lead to slower but stable optimization. + +.. tip:: + + We recommend the following parameters: -#. Create a virtual environment: + .. list-table:: + :align: center + :widths: 25 75 + :header-rows: 1 - .. code-block:: bash + * - Parameter + - Value + * - Policy Class + - ACT + * - KL Weight + - 10 + * - Chunk Size + - 100 + * - Batch Size + - 2 + * - Num of Epochs + - 3000 + * - Learning Rate + - 1e-5 - $ conda create -n aloha python=3.8.10 +Evaluation (ACT) +---------------- -#. Activate the virtual environment: +To evaluate a trained model, follow the steps below: - .. code-block:: bash +#. :ref:`operation/bringup_shutdown:bringup` the ALOHA - $ conda activate aloha +#. Configure the environment -Install Dependencies -^^^^^^^^^^^^^^^^^^^^ + .. code-block:: bash + + $ source /opt/ros/humble/setup.bash # Configure ROS system install environment + $ source interbotix_ws/install/setup.bash # Configure ROS workspace environment + $ source //bin/activate # Configure ALOHA Python environment + $ cd ~//act/ + +#. Run the evaluation script + + .. code-block:: bash + :emphasize-lines: 13-14 + + python3 imitate_episodes.py \ + --task_name aloha_stationary_dummy \ + --ckpt_dir \ + --policy_class ACT \ + --kl_weight 10 \ + --chunk_size 100 \ + --hidden_dim 512 \ + --batch_size 8 \ + --dim_feedforward 3200 \ + --num_epochs 2000 \ + --lr 1e-5 \ + --seed 0 \ + --eval \ + --temporal_agg + +.. note:: + + - The ``task_name`` argument should match one of the task names in the ``TASK_CONFIGS``, as configured in the :ref:`operation/data_collection:Task Creation` section. + - The ``ckpt_dir`` argument should match the correct relative directory location of the trained policy. + - The ``eval`` flag will set the script into evaluation mode. + - The ``temporal_agg`` is not required, but helps to smoothen the trajectory of the robots. + + +ACT++ Workflow (Aloha Mobile) +============================= + +Install Dependencies (ACT++) +---------------------------- Install the necessary dependencies inside your containerized environment: @@ -56,6 +233,7 @@ Install the necessary dependencies inside your containerized environment: $ pip install h5py $ pip install ipython $ pip install matplotlib + $ pip install modern_robotics $ pip install mujoco==2.3.7 $ pip install opencv-python $ pip install packaging @@ -65,26 +243,29 @@ Install the necessary dependencies inside your containerized environment: $ pip install rospkg $ pip install torch $ pip install torchvision + $ pip install transforms3d + $ pip install wandb -Clone Repository -================ - -Clone ACT if using Aloha Stationary +Clone Robomimic +--------------- .. code-block:: bash $ cd ~ - $ git clone https://github.com/Interbotix/act.git act_training_evaluation + $ git clone -b r2d2 https://github.com/ARISE-Initiative/robomimic.git + $ cd robomimic + $ pip install -e . -Clone ACT++ if using Aloha Mobile +Clone ACT++ Repository +---------------------- .. code-block:: bash $ cd ~ $ git clone https://github.com/Interbotix/act_plus_plus.git act_training_evaluation -Build and Install ACT Models -============================ +Build and Install ACT++ Models +------------------------------ .. code-block:: bash :emphasize-lines: 4 @@ -107,15 +288,14 @@ Build and Install ACT Models ├── LICENSE └── README.md - -Navigate to the ``detr`` directory inside the repository and install the detr module whihc contains the model definitions using the below command: +Navigate to the ``act_plus_plus`` directory inside the repository and install the required modules for model definitions by running the following command: .. code-block:: bash - $ cd /path/to/act/detr && pip install -e . + $ cd /path/to/act_plus_plus && pip install -e . -Training -======== +Training (ACT++) +---------------- To start the training, follow the steps below: @@ -142,7 +322,7 @@ To start the training, follow the steps below: $ cd /path/to/act/repository/ $ python3 imitate_episodes.py \ - --task_name aloha_stationary_dummy \ + --task_name aloha_mobile_dummy \ --ckpt_dir \ --policy_class ACT \ --kl_weight 10 \ @@ -150,7 +330,7 @@ To start the training, follow the steps below: --hidden_dim 512 \ --batch_size 8 \ --dim_feedforward 3200 \ - --num_epochs 2000 \ + --num_steps 2000 \ --lr 1e-5 \ --seed 0 @@ -162,7 +342,7 @@ To start the training, follow the steps below: - ``kl_weight``: Controls the balance between exploration and exploitation. - ``chunk_size``: Determines the length of the action sequence. K=1 is no action chunking and K=episode length is full open loop control. - ``batch_size``: Low batch size leads to better generalization and high batch size results in slower convergence but faster training time. - - ``num_epochs``: Too many epochs lead to overfitting; too few epochs may not allow the model to learn. + - ``num_steps``: Too many steps lead to overfitting; too few steps may not allow the model to learn. - ``lr``: Higher learning rate can lead to faster convergence but may overshoot the optima, while lower learning rate might lead to slower but stable optimization. @@ -185,13 +365,13 @@ To start the training, follow the steps below: - 100 * - Batch Size - 2 - * - Num of Epochs + * - Num of Steps - 3000 * - Learning Rate - 1e-5 -Evaluation -========== +Evaluation (ACT++) +------------------ To evaluate a trained model, follow the steps below: @@ -204,7 +384,7 @@ To evaluate a trained model, follow the steps below: $ source /opt/ros/humble/setup.bash # Configure ROS system install environment $ source interbotix_ws/install/setup.bash # Configure ROS workspace environment $ source //bin/activate # Configure ALOHA Python environment - $ cd ~//act/ + $ cd ~//act_plus_plus/ #. Run the evaluation script @@ -220,7 +400,7 @@ To evaluate a trained model, follow the steps below: --hidden_dim 512 \ --batch_size 8 \ --dim_feedforward 3200 \ - --num_epochs 2000 \ + --num_steps 2000 \ --lr 1e-5 \ --seed 0 \ --eval \ @@ -231,4 +411,12 @@ To evaluate a trained model, follow the steps below: - The ``task_name`` argument should match one of the task names in the ``TASK_CONFIGS``, as configured in the :ref:`operation/data_collection:Task Creation` section. - The ``ckpt_dir`` argument should match the correct relative directory location of the trained policy. - The ``eval`` flag will set the script into evaluation mode. - - The ``temporal_agg`` is not required, but helps to smoothen the trajectory of the robots. \ No newline at end of file + - The ``temporal_agg`` is not required, but helps to smoothen the trajectory of the robots. + +Major Differences Between ACT and ACT++ +======================================= + +- **Repository**: ACT uses `act`, while ACT++ uses `act_plus_plus`. +- **Robomimic Dependency**: Required for ACT++ but not for ACT. +- **Training Argument**: ACT uses `num_epochs`, while ACT++ uses `num_steps`. +- **Installation Directory**: For ACT, `pip install` is run inside `detr`, while for ACT++ it’s in the root directory.