아나콘다는 사용하지 않는다. AWSGPU+Ubuntu14.04+jupyter+theano+chainer+OpenCV3.1.0+cuDNN 환경 구축

환경



AWS GPU 인스턴스 사용
· 인스턴스 유형: g2.2xlarge
・AMI ID: ubuntu/images/hvm-ssd/ubuntu-trusty-14.04-amd64-server-20160627 (ami-2d39803a)

기본 설치


$ sudo apt-get update
$ sudo apt-get install build-essential cmake pkg-config
$ sudo apt-get install libjpeg-dev libtiff5-dev libjasper-dev libpng12-dev
$ sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev
$ sudo apt-get install libxvidcore-dev libx264-dev
$ sudo apt-get install libgtk2.0-dev
$ sudo apt-get install libatlas-base-dev gfortran
$ sudo apt-get -y install cmake git libgtk2.0-dev ocl-icd-opencl-dev
$ python --version
Python 2.7.6
$ sudo apt-get install python-dev python-pip
$ sudo apt-get install unzip
$ pip install --upgrade pip --user
$ pip install ipython pyzmq tornado --user
$ pip install jsonschema --user
$ pip install numpy matplotlib scipy --user

cuda 설치


$ wget http://developer.download.nvidia.com/compute/cuda/7.5/Prod/local_installers/cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb
$ sudo dpkg -i cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb
$ sudo apt-get update
$ sudo apt-get install cuda
$ echo 'export PATH=/usr/local/cuda-7.5/bin:$PATH' >> ~/.bashrc
$ echo 'export LD_LIBRARY_PATH=/usr/local/cuda-7.5/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
$ source .bashrc
$ sudo init 6
$ cat /proc/driver/nvidia/version
NVRM version: NVIDIA UNIX x86_64 Kernel Module  367.57  Mon Oct  3 20:37:01 PDT 2016
GCC version:  gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.3) 
$ dpkg -l | grep nvidia
ii  nvidia-352                                367.57-0ubuntu0.14.04.1                             amd64        Transitional package for nvidia-367
ii  nvidia-352-dev                            367.57-0ubuntu0.14.04.1                             amd64        Transitional package for nvidia-367-dev
ii  nvidia-352-uvm                            352.39-0ubuntu1                                     amd64        Transitional package for nvidia-352
ii  nvidia-367                                367.57-0ubuntu0.14.04.1                             amd64        NVIDIA binary driver - version 367.57
ii  nvidia-367-dev                            367.57-0ubuntu0.14.04.1                             amd64        NVIDIA binary Xorg driver development files
ii  nvidia-modprobe                           352.39-0ubuntu1                                     amd64        Load the NVIDIA kernel driver and create device files
ii  nvidia-opencl-icd-352                     367.57-0ubuntu0.14.04.1                             amd64        Transitional package for nvidia-opencl-icd-367
ii  nvidia-opencl-icd-367                     367.57-0ubuntu0.14.04.1                             amd64        NVIDIA OpenCL ICD
ii  nvidia-prime                              0.6.2                                               amd64        Tools to enable NVIDIA's Prime
ii  nvidia-settings                           352.39-0ubuntu1                                     amd64        Tool for configuring the NVIDIA graphics driver

cudnn 설치


  • cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb 얻는 방법
    htps : //에서 ゔぇぺぺr. 응아아. 코 m / kud
    에서 등록하고 다운로드 버튼을 누릅니다.Download cuDNN v4 (Feb 10, 2016), for CUDA 7.0 and later.를 선택하고 cuDNN v4 Library for Linux를 선택하여 다운로드하고 scp를 통해 /home/ubuntu로 파일을 전송하십시오.
  • $ cd ~
    $ tar -xvzf cudnn-7.0-linux-x64-v4.0-prod.tgz
    libcudnn.so libcudnn.so.4 libcudnn.so.4.0.7 libcudnn_static.aが入ってるディレクトリの絶対パスをLD_LIBRARY_PATHに追加
    $ cd cuda/lib64
    $ ls
    libcudnn.so  libcudnn.so.4  libcudnn.so.4.0.7  libcudnn_static.a
    $ pwd
    /home/ubuntu/cuda/lib64
    $ LD_LIBRARY_PATH=/home/ubuntu/cuda/lib64; export LD_LIBRARY_PATH
    $ cd ~
    $ sudo cp cuda/include/cudnn.h /usr/local/cuda/include
    $ sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
    $ sudo chmod a+r /usr/local/cuda/lib64/libcudnn*
    $ export PATH=/usr/local/cuda/bin:$PATH
    $ export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
    $ source .bashrc
    

    Jupyter 설치


    $ sudo apt-get remove --purge ipython
    $ sudo apt-get install libjpeg tk-dev
    $ pip install jupyter --user
    $ pip install wcwidth --user
    $ ~/.local/bin/jupyter notebook --generate-config
    $ vim ~/.jupyter/jupyter_notebook_config.py
    $ python
    >>> from notebook.auth import passwd
    >>> passwd()
    Enter password: 【なんらかのパスワードを入力】
    Verify password: 【なんらかのパスワードを入力】
    'sha1:84acf9ab4e38:5498f3245ea50214fd5bdd3aaa2050b33d5c5f91'
    >>> exit()
    $ vi ~/.jupyter/jupyter_notebook_config.py
    +c.NotebookApp.ip ='*'
    +c.NotebookApp.port = 8888
    +c.NotebookApp.password = u'sha1:84acf9ab4e38:5498f3245ea50214fd5bdd3aaa2050b33d5c5f91'
    $ echo 'export PATH=$PATH:~/.local/bin' >> ~/.bashrc
    $ source ~/.bashrc
    

    cuda 경로 연관


    $ echo 'export PATH=/usr/local/cuda-7.5/bin:$PATH' >> ~/.bashrc
    $ echo 'export LD_LIBRARY_PATH=/usr/local/cuda-7.5/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
    $ source .bashrc
    

    theano & chainer 설치



    ※ chainer의 인스톨을 cudnn의 인스톨보다 먼저 하면 GPU처리가 정상적으로 통과하지 않게 되므로 주의
    $ pip install theano --user
    $ pip install chainer --user
    

    opencv3.1.0 설치


    $ wget --no-check-certificate https://github.com/Itseez/opencv/archive/3.1.0.zip -O opencv-3.1.0.zip
    $ unzip opencv-3.1.0.zip
    $ cd opencv-3.1.0
    $ git clone --depth 1 https://github.com/Itseez/opencv_contrib.git opencv_contrib
    $ cd opencv_contrib
    $ git fetch origin --tags --depth 1
    $ git checkout 3.1.0
    $ sudo apt-get -y -qq install cmake git libgtk2.0-dev ocl-icd-opencl-dev qt5-default
    $ sudo apt-get install libhdf5-dev
    $ pip install h5py --user
    $ cd ..
    $ mkdir build
    $ cd build
    $ cmake -D CMAKE_BUILD_TYPE=RelWithDebugInfo -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D OPENCV_EXTRA_MODULES_PATH=../opencv_contrib/modules -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON  -D INSTALL_PYTHON_EXAMPLES=ON -D WITH_CUDA=ON -D ENABLE_PRECOMPILED_HEADERS=OFF -D CUDA_ARCH_BIN="3.0" -D CUDA_ARCH_PTX="" -D WITH_OPENGL=ON -D BUILD_EXAMPLES=ON -D WITH_QT=ON -D WITH_OPENGL=ON -D ENABLE_FAST_MATH=1 -D CUDA_FAST_MATH=1 ..
    $ make -j 4
    $ sudo make install -j 4
    $ sudo ldconfig
    $ echo 'export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH' >> ~/.bashrc
    $ source ~/.bashrc
    $ pkg-config --modversion opencv
    3.1.0
    

    OpenCV 테스트


    $ sudo ln /dev/null /dev/raw1394
    $ cd ~
    $ mkdir test
    $ cd test
    $ wget http://www.sonicjapan.co.jp/sample/download/AVI-352x240-2sec.AVI
    $ python
    >>> import numpy as np
    >>> import cv2
    >>> cap = cv2.VideoCapture('./AVI-352x240-2sec.AVI')
    >>> cap.read()
    

    jupyter notebook에서 동영상 읽기 확인


    $ cd ~
    $ jupyter notebook
    先ほどのtestディレクトリへ移動してnotebookを作り、下記実行
    import numpy as np
    import cv2
    cap = cv2.VideoCapture('./AVI-352x240-2sec.AVI')
    cap.read()
    



    theano의 GPU 처리 테스트


    $ cd ~
    $ git clone https://github.com/aidiary/deep-learning-theano
    $ cd ~/deep-learning-theano/scripts
    $ THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python convnet.py
    Using gpu device 0: GRID K520
    ....
    Optimization complete.
    Best validation score of 0.910000 % obtained at iteration 16400, with test performance 0.930000 %
    The code for file convnet.py ran for 39.85m
    

    chainer의 GPU 처리 테스트


    $ git clone https://github.com/pfnet/chainer.git
    $ python ~/chainer/examples/mnist/train_mnist.py -g 0
    

    좋은 웹페이지 즐겨찾기