Skip to content
This repository has been archived by the owner on Jun 4, 2021. It is now read-only.

1.1 Install the package

Jump to bottom
piek edited this page Nov 17, 2020 · 3 revisions

1. Python 2.7 (Required) & Naoqi Python SDK (Optional)

This project builds onto the Naoqi Python SDK by SoftBank Robotics. This allows you to run our code on Pepper / Nao Robots. Support for the Naoqi backend is included in this repository. However Naoqi is not a requirement for other backends (for instance your own pc).

Download the Python SDK (version 2.5.10) from the Softbank robotics download page. Please refer to the Naoqi Python SDK install guide for more information (note that the download links on this page are outdated). The SDK, and by extension this repository, is only available for 32-bit Python 2.7 on Windows and 64-bit Python 2.7 on Linux and OSX, so make sure you grab the right Python version if you intent to run code on Pepper / Nao Robots.

  • Please make sure the PYTHONPATH environment variable reflects the location of the SDK and you're using Python 2.7
  • Please make sure you install Python as a Framework on OSX or else pynaoqi will complain. See next section for more details.

Virtual environments

In general, it is good practice to separate projects via virtual environments. Naoqi is a particularly annoying package that does not play along many set ups. specifically, it needs this version of Python 2.7.10 So far, this has been tried:

  • Unsuccessful
    • Anaconda: Anaconda does not support Python 2.7.10 anymore (you can read about this here). You can get previous unsupported packages either via conda-forge or restoring the free channel, which is not recommended. Both approaches were tried and did not manage to run the Naoqi SDK.
  • Successful
    • Pip and virtualenv:
      • Download Python manually via the link. To fully understand how this interacts with your existing Python environment please read this. This will download two things: a) A Python folder in your applications folder, and b) A framework which includes the Python executable. It is important here to check that the path looks something like this /Library/Frameworks/Python.framework.
      • Download NaoqiSDK and put it in the project folder.
      • Run the following commands
pip install "virtualenv<20.0"
cd {PATH_TO_YOUR_PROJECT}/pepper
virtualenv -p /Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7 venv
source venv/bin/activate
export DYLD_LIBRARY_PATH=${DYLD_LIBRARY_PATH}:{PATH_TO_YOUR_PROJECT}/pynaoqi-python2.7-2.5.5.5-mac64/lib
export PYTHONPATH=${PYTHONPATH}:{PATH_TO_YOUR_PROJECT}/pynaoqi-python2.7-2.5.5.5-mac64/lib/python2.7/site-packages
python
import naoqi

A series of errors may appear, like this

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/selbaez/Documents/leolani/pynaoqi-python2.7-2.5.5.5-mac64/lib/python2.7/site-packages/naoqi.py", line 64, in <module>
    import inaoqi
  File "/Users/selbaez/Documents/leolani/pynaoqi-python2.7-2.5.5.5-mac64/lib/python2.7/site-packages/inaoqi.py", line 28, in <module>
    _inaoqi = swig_import_helper()
  File "/Users/selbaez/Documents/leolani/pynaoqi-python2.7-2.5.5.5-mac64/lib/python2.7/site-packages/inaoqi.py", line 24, in swig_import_helper
    _mod = imp.load_module('_inaoqi', fp, pathname, description)
ImportError: dlopen(/Users/selbaez/Documents/leolani/pynaoqi-python2.7-2.5.5.5-mac64/lib/python2.7/site-packages/_inaoqi.so, 2): Library not loaded: @loader_path/libalerror.dylib
  Referenced from: /Users/selbaez/Documents/leolani/pynaoqi-python2.7-2.5.5.5-mac64/lib/python2.7/site-packages/_inaoqi.so
  Reason: no suitable image found.  Did find:
	/Users/selbaez/Documents/leolani/pynaoqi-python2.7-2.5.5.5-mac64/lib/libalerror.dylib: code signature in (/Users/selbaez/Documents/leolani/pynaoqi-python2.7-2.5.5.5-mac64/lib/libalerror.dylib) not valid for use in process using Library Validation: library load disallowed by system policy

You will need to accept to opening the file in the Mac, Security settings, like this. This cycle must be done 11 times, for 11 different files like libalvalue.dylib, libalerror.dylib, etc. Please continue doing this until you can run import naoqi without errors.

2. Google Cloud Services

his projects makes use of Google Cloud services for Speech Recognition & Generation. This is a commercial service, although a trail for a free year is available. After following instructions to make a cloud project on their website, download your key as google_cloud_key.json and place it in the root of this repo, it is then automatically added as environment variable at runtime.

Google Cloud Speech-to-Text

The Google Cloud Speech-to-Text API is used as Speech Recognition solution for this project. Please refer to their website for licencing and installation instructions. It is of course possible to use a different Automatic Speech Recognition (ASR) solution, if you wish, but we found it difficult to find a good performing open source solution... Call pip install google-cloud-speech in order to install the required Python libraries.

Google Cloud Text-To-Speech

The Google Cloud Text-To-Speech API is used a Text to Speech solution when running applications on your PC, instead of on the robot. Please, again, refer to their website for licencing and installation instructions. Call pip install google-cloud-TextToSpeech playsound in order to install the required Python libraries.

  • If your platform happens to be Mac, and the playsound installation complains about the cairo dependency, try running pip install -U PyObjC.
  • If your platform happens to be Mac, and the playsound installation complains about the 'portaudio.h' file not found, try running brew install portaudio before trying pip install pyaudio as explained here.
  • If your platform happens to be Mac and if the "brew install" did not work either, try this one: pip install --global-option='build_ext' --global-option='-I/usr/local/include' --global-option='-L/usr/local/lib' pyaudio

3. OpenFace & Docker

Face recognition in this project is done using the open source OpenFace project (Site, Git), which is, to put it mildly, difficult to install; Probably even impossible on Windows. Luckily, the developers were so kind to provide a Docker image, which can be obtained using docker pull bamos/openface. Once Docker is installed and the bamos/openface dependency is pulled, this repo will automatically interface with it. If you're planning to run Docker on a Windows machine, make sure you have Windows 10 64bit: Pro, Enterprise or Education, since Docker relies on Microsoft Hyper-V, which is only available on these versions of Windows. A possible workaround could involve Docker Toolbox, which does work on lower Windows versions and is able to run containers. As of now, we did not test this.

4. Object Recognition & Pepper Tensorflow

In order to use the Inception and COCO models natively within this project, you need to clone pepper's sister-project: pepper_tensorflow, which includes the Tensorflow services within Python 3.6. These services need to run as a separate process either locally or remotely next to the main application. A Socket-interface is implemented bridging the Python 2 / 3 gap. Since Tensorflow requires Python 3, this is our way of using Tensorflow in an otherwise Python 2.7 repo.

We recommend creating two virtual environments, one Python 2.7 environment for the main repo (pepper) and an Python 3.6 environment for the Tensorflow repo (pepper_tensorflow)

The main script to run here is object_recognition.py, which finds objects in images and annotates them with bounding boxes. In fact, any model from the Tensorflow Detection Model Zoo can be used here! object_recognition.py gives hints on how to do this. To include other object detection models, you'll have to download the data from the Model Zoo yourself first!

5. NLTK for Natural Language Understanding

For analyzing utterances, this project relies on NLTK. The current version to use is nltk==3.4. Its data dependencies are automatically installed once you launch the program for the first time. For Named Entity Recognition, you will need to have Java installed on your machine. Please make sure it is callable from the command line (and fix your environment variables if not).

6. GraphDB for Knowledge Representation

In this project, knowledge is represented in the form of triples and stored in a triple store. To install the necessary Python dependencies, you need to run pip install rdflib iribaker SPARQLWrapper. Additionally, you have to install GraphDB. Please follow the instructions to download, the free version will suffice.

GraphDB's UI can be accessed through http://localhost:7200/. From the UI, you will need to set up a new repository called leolani. Don't forget to connect to the repository when you start GraphDB.

7. Wolfram Alpha

This project makes use of the Wolfram Alpha Spoken Results API A free (academic) licence is available that allows 2000 queries a month (which is plenty, in our experience). Please create an account and place a file tokens.json, with the following information to the root of this project (right next to the google_cloud_key.json):

{
  "wolfram": <YOUR KEY>
}

This will enable Wolfram Alpha into your Robot applications.

8. Other Python Dependencies

The project requirements are listed in requirements.txt and can be easily installed via pip.

On Windows, an OpenCV binary needs to be downloaded. pip install opencv-python will be sufficient for OSX platforms.

Clone this wiki locally