This repo is a clone of my dissertation, which was submitted to the University of Liverpool as required for my Masters in Computer Science. Within it, two optical-tactile-based-sensors (GelSight) are simulated and appended to a UR5 arm. Mujoco is used as the simulation environment - a physics extension to OpenAi's gym. This platform was utilised to construct the environments Reach, Push, Slide and PickandPlace (similar to those found within gym). Which in turn act as a test-bed for training a deep reinforcement learning agent using both Double Deep Q-Learning (DDQN) and Hierarchical Double Deep Q-Learning (HDDQN).
- install mujoco - https://mujoco.org/
- git clone the repo
- cd into repo_path/Test_bed
- run python main.py
- download any missing packages
| Parameter | Description |
|---|---|
| env | Choose an environment to run |
| render | Draw the environment as it runs? |
| random_spawns | Randomize the placement of objects in the environment |
| gelsight_dims | Resolution of the internal GelSight camera |
| envcam_dims | Resolution of the agent's (and yours if render=True) view of the environment |
| image_rescale | Should the images be scaled down? |
| image_grayscale | Should the images be grayscale? |
| image_normalise | Should the images be normalized? |
| framestack | Should the frames be stacked when passed to the neural network |
| k_frames | How many frames should be in the stack |
| agent | The deep reinforcement learning algorithm to run |
| mode | Are we training or testing? |
| number_of_episodes | Episodes that must be completed before the agent and environment stop |
| model_weights0 | Are you loading any weights for the first neural network? |
| model_weights1 | Are you loading any weights for the second neural network (hddqn)? |
| replay_buffer_size | How many memories do you want to store? |
| batch_size | How many states do you want to select at a time to learn from |
| alpha_lr | The learning rate |
| beta_lr | The learning rate for the second neural network (hddqn) |
| nn_dense | The density of layers in the neural network post concatenation - following conv2d and any kinematics |
| nn_num_dense | The density of layers for any kinematic inputs |
| ext_nn_dense | (hddqn) The density of layers in the neural network post concatenation - following conv2d and any kinematics |
| ext_nn_num_dense | (hddqn) The density of layers for any kinematic inputs |
| transfer_model | Transfer learning from another model - conv2d only |
| type_ | What type of data should be included? [vision, tactile, kinematics, raw] |
| extrinsic_type2_ | (hddqn) What type of data should be included? [vision, tactile, kinematics, raw] |
| seed | Randomization seed |
- create requirements.txt
- create gitignore
- refactor and optimise code
