Summary

This repo is only a draft demo based on Mini Pupper 2 with ROS2 Humble now. To try it, you need add your own AWS and chatGPT account info to config file(gpt_status/gpt_status/gpt_config.py).

The basic process is,

your voice ---> Mini Pupper 2 record by Mic x2 ---> translate voice to text by AWS service ---> chatGPT API ---> translate text to voice by AWS service ---> Mini Pupper 2 voice Playback & Movement & emotion.

It's a ROS2 Humble interface demo designed to empower Mini Pupper 2 with voice, motion, and emotion control capabilities. This package enables you to leverage LLM-based features, such as GPT-4 & ChatGPT, to enhance the functionality of their robotic applications within the ROS2 ecosystem. It provides a dynamic solution for creating engaging and interactive experiences.

Installation

One-click Installation

Mini Pupper 2 and Ubuntu 22.04 + ROS 2 Humble is required. Please follow the installation document here

To install with one command, connect to your Mini Pupper 2, be certain to tell ssh to allow X11 forwarding if you want to run demo 1.

ssh -o ForwardX11=yes ubuntu@<Your Mini Pupper 2 IP address>

and then run the following command:

wget -O $HOME/install.sh https://raw.githubusercontent.com/mangdangroboticsclub/chatgpt-minipupper2-ros2-humble/main/install.sh && sudo chmod +x $HOME/install.sh && bash $HOME/install.sh && rm $HOME/install.sh

After the one-click Installation, demo 1 Simple robot GPT call on the PC side will run automatically, if you want to run other demos, please modify the configuration file according to Step4 of Manual Installation

Manual Installation

If you want to install manually, follow the steps below.

Step 1: Clone the repo

cd <your_ws>/src
git clone https://github.com/mangdangroboticsclub/chatgpt-minipupper2-ros2-humble.git

Step 2: Install dependencies

cd <your_ws>/src/chatgpt-minipupper2-ros2-humble
sudo chmod +x dependencies_install.sh
. dependencies_install.sh # Install dependencies

Step 3: Build the repo

cd <your_ws>
rosdep install --from-paths src --ignore-src -r -y
colcon build --symlink-install

Step 4: Configuration

To use the gpt4_ros2 package, follow these steps:

4.1 Set up AWS

4.1.1 Create an S3 bucket

1. Create an AWS account.
2. Go to the AWS Console Amazon S3 page.
3. Click Create bucket.
4. Enter a Bucket name and choose an AWS Region.
5. Click Create bucket.

4.1.2 Set up an IAM user

1. Go to the AWS Console Amazon IAM page.
2. Click Users under the IAM resources section.
3. Click Add user.
4. Enter a User name.
5. Under Set permissions, click Attach existing policies directly and search for the following policies:
   • AmazonPollyFullAccess
   • AmazonTranscribeFullAccess
   • AmazonS3FullAccess
6. Add the selected policies to the user.
7. Click Next, review the Permissions summary and any other information.
8. Click Create user.

4.2 Set up OpenAI API

1. Create an account on OpenAI.
2. Click on the user icon in the upper-right corner.
3. Click View API keys.
4. Click Create new secret key.
5. Enter a name and click Create secret key.
6. Copy your secret key and save it securely.

4.3 Configure and build the package

1. Navigate to <your_ws>/src/chatgpt-minipupper2-ros2-humble/gpt_status/gpt_status/gpt_config.py.

cd <your_ws>/src/chatgpt-minipupper2-ros2-humble/gpt_status/gpt_status

2. Set your desired configurations, such as the GPT-4 or GPT-3.5-turbo model, system_prompt, and other attributes. Fill in the relevant configuration details for AWS and OpenAI that you obtained earlier.

sudo nano gpt_config.py

4.4 Modify the gpt_robot package code [optional]

If you wish to use GPT for your own robots, modify the contents of the gpt_robot package, which configures physical or virtual robots.

We encourage you to customize the GPTConfig class to tailor the functionality of this ROS2 wrapper for GPT-4 and ChatGPT (GPT-3.5) according to your specific needs. To do this, simply modify the values in the code snippet to suit your requirements:

GPT-4 is currently in a limited beta and only accessible to those who have been granted access. Please join the waitlist to get access when capacity is available.

Feel free to adjust the parameters, such as temperature, max_tokens, and top_p, to influence the behavior of the GPT model. You can also customize the system_prompt and user_prompt strings to create unique and engaging interactions with your robot.

By personalizing these settings, you can create a one-of-a-kind experience tailored to your specific robotic application. Enjoy experimenting and discovering new possibilities!

Usage

Demo 1: Simple GPT call on the PC

If you want to simply try this service, configure your OpenAI API and system_prompt in gpt_status/gpt_config.py. Then, try:

# Terminal 1
ros2 run gpt_main gpt_ros2_server

# Terminal 2
ros2 run gpt_main gpt_ros2_client

Demo 2: GPT service on Mini Pupper 2

After configuring everything, run the below commands on Mini Pupper 2:

# Terminal 1 Bringup mini pupper
. ~/ros2_ws/install/setup.bash
ros2 launch mini_pupper_bringup bringup.launch.py

# Terminal 2 Bringup GPT
ros2 launch gpt_bringup gpt_bringup_launch.py mini_pupper:=True

License

This project is licensed under the Apache-2.0 License.

Copyright 2023 Mangdang
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
    http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.