Setup Neuron Environment

Set up Inf1 instance as a development environment for compiling pre-trained machine learning models and a deployment environment for running the compiled models.

You can select Deep Learning AMI such as Ubuntu 18.x, or Amazon Linux 2-based AMI.

In this workshop, the development environment and the deployment environment will be implemented on the same inf1.2xlarge instance. If you need to compile a larger model, we recommend using an instance of c5.4xlarge or larger as your development environment.

Step 1. Launch an Inf1 Instance as a development and deployment environment

Please follow the instructions at launch an Amazon EC2 Instance to Launch inf1.2xlarge instance and select Deep Learning AMI (Ubuntu 18.04) version 44.

Step 2. Set up pre-installed Neuron development environment

We will follow the steps to use the pre-installed Neuron SDK in the Deep Learning AMI. If you want to install the Neuron SDK individually, please refer to here.

Software tools and packages are updated frequently, so run the update process.

sudo apt-get update
sudo apt-get install aws-neuron-dkms
sudo apt-get install aws-neuron-runtime-base
sudo apt-get install aws-neuron-runtime
sudo apt-get install aws-neuron-tools
sudo apt-get install tensorflow-model-server-neuron

If you see the following error while installing apt-get, wait a minute or so for the background update to complete, then try apt-get install again.

E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable)
E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?

Make sure you have the latest development environment installed.

dpkg -l | grep neuron

The latest version as of June 1, 2021 is as follows. You can see that the Neuron runtime, Neuron tools, etc. are installed.

ii  aws-neuron-dkms                        1.5.0.0                             amd64        aws-neuron driver in DKMS format.
ii  aws-neuron-runtime                     1.5.0.0                             amd64        Neuron Runtime Driver and Inference Server
ii  aws-neuron-runtime-base                1.5.1.0                             amd64        neuron_base built using CMake
ii  aws-neuron-tools                       1.6.1.0                             amd64        Neuron profile and debug tools
ii  tensorflow-model-server-neuron         1.15.0.1.4.0.0                      all          TensorFlow Serving ModelServer

Launch the pre-installed Tensorflow Neuron development environment on the Deep Learning AMI and run the update process.

source activate aws_neuron_tensorflow_p36
pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com
pip install --upgrade tensorflow-neuron tensorboard-plugin-neuron neuron-cc

Check the installed Tensorflow Neuron development environment.

pip list | grep neuron

The latest version as of June 1, 2021 is as follows. You can see that the Neuron compiler, Tensorflow Neuron, etc. are installed.

neuron-cc                          1.4.1.0+737cbb69a
tensorboard-plugin-neuron          2.1.0.0
tensorflow-neuron                  1.15.5.1.4.0.0

Software tools and packages are updated frequently, so it’s important to make sure you’re always up to date.

Starting with Neuron SDK 1.14.0, we will no longer support conda packages to install Neuron SDK framework in DLAMI. Please refer to blog.