Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes to get builds working again #85

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Fixes to get builds working again #85

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fa6dfe8
Commenting out Caffe and OpenCV from README
opencoca Mar 28, 2018
de9050f
Upgrade python for SSL fix
opencoca Mar 28, 2018
88b2758
Merge branch 'master' of https://github.com/opencoca/dl-docker
opencoca Mar 29, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
128 changes: 86 additions & 42 deletions Dockerfile.gpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,16 @@ ARG LASAGNE_VERSION=v0.1
ARG TORCH_VERSION=latest
ARG CAFFE_VERSION=master

ARG PYTHON_BASEDEPS="build-essential python-pip"
ARG PYTHON_BUILDDEPS="libbz2-dev \
libc6-dev \
libgdbm-dev \
libncursesw5-dev \
libreadline-gplv2-dev \
libsqlite3-dev \
libssl-dev \
tk-dev"

#RUN echo -e "\n**********************\nNVIDIA Driver Version\n**********************\n" && \
# cat /proc/driver/nvidia/version && \
# echo -e "\n**********************\nCUDA Version\n**********************\n" && \
Expand Down Expand Up @@ -87,6 +97,24 @@ RUN apt-get update && apt-get install -y \
# Link BLAS library to use OpenBLAS using the alternatives mechanism (https://www.scipy.org/scipylib/building/linux.html#debian-ubuntu)
update-alternatives --set libblas.so.3 /usr/lib/openblas-base/libblas.so.3


ARG PYTHON_TARFILE="Python-2.7.9.tgz"
ARG PYTHON_TARHOST="https://www.python.org/ftp/python/2.7.9"
ARG PYTHON_SRCDIR="Python-2.7.9"

RUN apt-get update
RUN apt-get install -y ${PYTHON_BASEDEPS} ${PYTHON_BUILDDEPS}

RUN wget "${PYTHON_TARHOST}/${PYTHON_TARFILE}"
RUN tar xvf ${PYTHON_TARFILE}

RUN cd ${PYTHON_SRCDIR} && \
./configure && \
make && \
make install



# Install pip
RUN curl -O https://bootstrap.pypa.io/get-pip.py && \
python get-pip.py && \
Expand All @@ -109,28 +137,41 @@ RUN apt-get update && apt-get install -y \
python-matplotlib \
python-pandas \
python-sklearn \
python-sympy \
&& \
python-sympy
RUN apt-get update && apt-get install -y \
libffi-dev libssl-dev && \
apt-get clean && \
apt-get autoremove && \
rm -rf /var/lib/apt/lists/*

# Install other useful Python packages using pip
RUN pip --no-cache-dir install --upgrade ipython && \
pip --no-cache-dir install \
Cython \
ipykernel \
jupyter \
path.py \
Pillow \
pygments \
six \
sphinx \
wheel \
zmq \
&& \
python -m ipykernel.kernelspec
RUN pip install --upgrade pip
RUN pip install pyopenssl ndg-httpsclient pyasn1
RUN pip --no-cache-dir install --upgrade ipython
RUN pip --no-cache-dir install Cython
RUN pip --no-cache-dir install path.py
RUN pip --no-cache-dir install Pillow
RUN pip --no-cache-dir install pygments
RUN pip --no-cache-dir install six
RUN pip --no-cache-dir install sphinx
RUN pip --no-cache-dir install wheel
RUN pip --no-cache-dir install zmq

#TODO move UP
RUN wget https://www.openssl.org/source/openssl-1.0.2n.tar.gz
RUN tar -xvzf openssl-1.0.2n.tar.gz
RUN cd openssl-1.0.2n && ./config && sudo make && sudo make install
RUN ln -sf /usr/local/ssl/bin/openssl `which openssl`

RUN sleep 1
RUN openssl version -v

RUN pip --no-cache-dir install ipykernel
RUN pip --no-cache-dir install jupyter
RUN python -m ipykernel.kernelspec

#TODO take out if ipykernel gets installed
#RUN false

# Install TensorFlow
RUN pip --no-cache-dir install \
Expand All @@ -154,22 +195,25 @@ RUN apt-get update && apt-get install -y \
apt-get autoremove && \
rm -rf /var/lib/apt/lists/*

# Install Caffe
RUN git clone -b ${CAFFE_VERSION} --depth 1 https://github.com/BVLC/caffe.git /root/caffe && \
cd /root/caffe && \
cat python/requirements.txt | xargs -n1 pip install && \
mkdir build && cd build && \
cmake -DUSE_CUDNN=1 -DBLAS=Open .. && \
make -j"$(nproc)" all && \
make install

# Set up Caffe environment variables
ENV CAFFE_ROOT=/root/caffe
ENV PYCAFFE_ROOT=$CAFFE_ROOT/python
ENV PYTHONPATH=$PYCAFFE_ROOT:$PYTHONPATH \
PATH=$CAFFE_ROOT/build/tools:$PYCAFFE_ROOT:$PATH

RUN echo "$CAFFE_ROOT/build/lib" >> /etc/ld.so.conf.d/caffe.conf && ldconfig
### Install Caffe
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You just removed all the Caffe and OpenCV, it's that ok ?

#RUN git clone -b ${CAFFE_VERSION} --depth 1 https://github.com/BVLC/caffe.git /root/caffe
#RUN cd /root/caffe && \
# cat python/requirements.txt | xargs -n1 -L 1 pip install
#RUN pip install -r /root/caffe/python/requirements.txt
#RUN cd /root/caffe && \
# mkdir build && cd build && \
# cmake -DUSE_CUDNN=1 -DBLAS=Open .. && \
# make -j"$(nproc)" all
#RUN cd /root/caffe/build && \
# make install
#
## Set up Caffe environment variables
#ENV CAFFE_ROOT=/root/caffe
#ENV PYCAFFE_ROOT=$CAFFE_ROOT/python
#ENV PYTHONPATH=$PYCAFFE_ROOT:$PYTHONPATH \
# PATH=$CAFFE_ROOT/build/tools:$PYCAFFE_ROOT:$PATH
#
#RUN echo "$CAFFE_ROOT/build/lib" >> /etc/ld.so.conf.d/caffe.conf && ldconfig


# Install Theano and set up Theano config (.theanorc) for CUDA and OpenBLAS
Expand Down Expand Up @@ -219,16 +263,16 @@ RUN luarocks install nn && \
cd iTorch && \
luarocks make

# Install OpenCV
RUN git clone --depth 1 https://github.com/opencv/opencv.git /root/opencv && \
cd /root/opencv && \
mkdir build && \
cd build && \
cmake -DWITH_QT=ON -DWITH_OPENGL=ON -DFORCE_VTK=ON -DWITH_TBB=ON -DWITH_GDAL=ON -DWITH_XINE=ON -DBUILD_EXAMPLES=ON .. && \
make -j"$(nproc)" && \
make install && \
ldconfig && \
echo 'ln /dev/null /dev/raw1394' >> ~/.bashrc
#TODO# Install OpenCV
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same as above.

#RUN git clone --depth 1 https://github.com/opencv/opencv.git /root/opencv && \
# cd /root/opencv && \
# mkdir build && \
# cd build && \
# cmake -DWITH_QT=ON -DWITH_OPENGL=ON -DFORCE_VTK=ON -DWITH_TBB=ON -DWITH_GDAL=ON -DWITH_XINE=ON -DBUILD_EXAMPLES=ON .. && \
# make -j"$(nproc)" && \
# make install && \
# ldconfig && \
# echo 'ln /dev/null /dev/raw1394' >> ~/.bashrc

# Set up notebook config
COPY jupyter_notebook_config.py /root/.jupyter/
Expand Down
6 changes: 2 additions & 4 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,23 +3,21 @@ Here are Dockerfiles to get you up and running with a fully functional deep lear

If you are not familiar with Docker, but would still like an all-in-one solution, start here: [What is Docker?](#what-is-docker). If you know what Docker is, but are wondering why we need one for deep learning, [see this](#why-do-i-need-a-docker)

## Update: I've built a quick tool, based on dl-docker, to run your DL project on the cloud with zero setup. You can start running your Tensorflow project on AWS in <30seconds using Floyd. See [www.floydhub.com](https://www.floydhub.com). It's free to try out.
### Happy to take feature requests/feedback and answer questions - mail me [email protected].

## Specs
This is what you get out of the box when you create a container with the provided image/Dockerfile:
* Ubuntu 14.04
* [CUDA 8.0](https://developer.nvidia.com/cuda-toolkit) (GPU version only)
* [cuDNN v5](https://developer.nvidia.com/cudnn) (GPU version only)
* [Tensorflow](https://www.tensorflow.org/)
* [Caffe](http://caffe.berkeleyvision.org/)
* ~~[Caffe](http://caffe.berkeleyvision.org/)~~
* [Theano](http://deeplearning.net/software/theano/)
* [Keras](http://keras.io/)
* [Lasagne](http://lasagne.readthedocs.io/en/latest/)
* [Torch](http://torch.ch/) (includes nn, cutorch, cunn and cuDNN bindings)
* [iPython/Jupyter Notebook](http://jupyter.org/) (including iTorch kernel)
* [Numpy](http://www.numpy.org/), [SciPy](https://www.scipy.org/), [Pandas](http://pandas.pydata.org/), [Scikit Learn](http://scikit-learn.org/), [Matplotlib](http://matplotlib.org/)
* [OpenCV](http://opencv.org/)
* ~~[OpenCV](http://opencv.org/)~~
* A few common libraries used for deep learning

## Setup
Expand Down