User:Jeblad/TensorFlow

To make an efficient environment for development it might be necessary to tweak and adapt the setup. That imply diverging versions for the various libraries, and even libraries that should not be defined on the computer as such. To make this possible either ''virtualenv'' or ''vagrant'' can be used. To make this possible either ''virtualenv'' or ''vagrant'' can be used. CUDA and TensorFlow is no different in this respect.

Virtualenv on main computer

Of some reason you want to run the CUDA and/or TensorFlow on the bare metal. Especially note that PCI passthrough must work vor Vagrant to be used, and if it does not work then this is the option of choice.

Environment at main computer

Installation of CUDA:

1. Open "System Settings"
2. Open "Software and Updates"
3. Open "Additional Drivers"
4. Select one of the Nvidia Drivers and click "Apply Changes"
5. Reboot the system.
6. Open a terminal window and type nvidia-smi. This will identify the graphics card.

CUDA

Pre-install actions:

$ sudo apt-get install linux-headers-$(uname -r)

Installation Instructions:

$ sudo dpkg -i cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64.deb
$ sudo apt-get update
$ sudo apt-get install cuda

Post-install actions: Add the following to .profile, but note the versioning, it must match the names in use!

# additional exports
export PATH=/usr/local/cuda/bin:/usr/local/cuda-8.0/bin:${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda-8.0/lib64:${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

Note the PATH, it will generate a double colon, and that is a bit dangerous. Later on that will create a warning.

Install cuDNN, note this is behind a member wall:

$ cd <installpath>
$ export LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH

Add <installpath> to your build and link process by adding -I<installpath> to your compile line and -L<installpath> -lcudnn to your link line, or simply put it into the cuda folder.

Samples

The samples are found at /usr/local/cuda/samples if the install is the usual deb-package for Ubuntu. It is although created for "nvidia-367" and not "nvidia-375" which is the latest as of this wring (10:38, 8 May 2017 (UTC)). To update the code (preferably on a local copy) do a

$ find . -type f -execdir sed -i 's/UBUNTU_PKG_NAME = "nvidia-367"/UBUNTU_PKG_NAME = "nvidia-375"/g' '{}' \;

and then a simple make.

Failing to update the samples will give a missing libnvcuvid.

virtualenv

Install virtualenv

$ sudo apt-get install python-pip python-dev python-virtualenv

It is not quite clear if this should be a python 3 specific version, I have no problem with it so far.

Now create a working directory, typically something like tensorflow, and create the virtualenv for the directory, and then activate

$ virtualenv --system-site-packages tensorflow
$ source tensorflow/bin/activate

Note that this only works with python 2, not with python 3!

TensorFlow

Do a cd tensorflow.

Play it safe and setup both python 2.7 and 3.5?

$ sudo apt-get install python-pip python-dev
$ sudo apt-get install python3-pip python3-dev

Assuming there is a gpu in the machine

$ pip3 install tensorflow-gpu

Fill in the blanks

$ sudo apt-get update && sudo apt-get upgrade

Virtualization of PCI hardware for Vagrant

Your PC must satisfy the following:

1. Your motherboard has an IOMMU unit.
2. Your CPU supports the IOMMU.
3. The IOMMU is enabled in the BIOS.
4. The VM must run with VT-x/AMD-V and nested paging enabled.
5. Your Linux kernel was compiled with IOMMU support. The PCI stub driver is required as well.
6. Your Linux kernel recognizes and uses the IOMMU unit. Search for DMAR and PCI-DMA in kernel boot log.

Environment at host

mkdir tensorspace
cd tensorspace
vagrant init bento/ubuntu-16.10
vagrant box update
vagrant up --provider virtualbox
vagrant plugin install vagrant-vbguest
vagrant reload

Now edit the nano Vagrantfile and add new limits for memory and cpus.

vagrant reload

Environment at client

There are additional notes for CUDA at Ubuntus help pages.

CUDA

Pre-install actions:

$ sudo apt-get install linux-headers-$(uname -r)

Installation Instructions:

$ sudo dpkg -i cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64.deb
$ sudo apt-get update
$ sudo apt-get install cuda

I put the deb in my vagrant dir on the host, and then referred i as sudo dpkg -i /vagrant/cuda-repo-ubuntu1604-8-0-local-ga2_8.0.61-1_amd64.deb inside the client.

Post-install actions: Add the following to .profile, but note the versioning, it must match the names in use!

# additional exports
export PATH=/usr/local/cuda/bin:/usr/local/cuda-8.0/bin:${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/cuda-8.0/lib64:${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

Install cuDNN, note this is behind a member wall:

$ cd <installpath>
$ export LD_LIBRARY_PATH=`pwd`:$LD_LIBRARY_PATH

Add <installpath> to your build and link process by adding -I<installpath> to your compile line and -L<installpath> -lcudnn to your link line.

TensorFlow

Play it safe and setup both python 2.7 and 3.5?

$ sudo apt-get install python-pip python-dev
$ sudo apt-get install python3-pip python3-dev

$ pip3 install tensorflow-gpu

$ sudo apt-get update && sudo apt-get upgrade

Manual testing

Open the interactive shell with python3 and run

import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))