Set up Inf1 instance as a development environment for compiling pre-trained machine learning models and a deployment environment for running the compiled models.
You can select Deep Learning AMI such as Ubuntu 18.x, or Amazon Linux 2-based AMI.
In this workshop, the development environment and the deployment environment will be implemented on the same inf1.2xlarge instance. If you need to compile a larger model, we recommend using an instance of c5.4xlarge or larger as your development environment.
We will follow the steps to use the pre-installed Neuron SDK in the Deep Learning AMI. If you want to install the Neuron SDK individually, please refer to here.
Software tools and packages are updated frequently, so run the update process.
sudo apt-get update sudo apt-get install aws-neuron-dkms sudo apt-get install aws-neuron-runtime-base sudo apt-get install aws-neuron-runtime sudo apt-get install aws-neuron-tools sudo apt-get install tensorflow-model-server-neuron
If you see the following error while installing apt-get, wait a minute or so for the background update to complete, then try apt-get install again.
E: Could not get lock /var/lib/dpkg/lock-frontend - open (11: Resource temporarily unavailable) E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?
Make sure you have the latest development environment installed.
dpkg -l | grep neuron
The latest version as of June 1, 2021 is as follows. You can see that the Neuron runtime, Neuron tools, etc. are installed.
ii aws-neuron-dkms 220.127.116.11 amd64 aws-neuron driver in DKMS format. ii aws-neuron-runtime 18.104.22.168 amd64 Neuron Runtime Driver and Inference Server ii aws-neuron-runtime-base 22.214.171.124 amd64 neuron_base built using CMake ii aws-neuron-tools 126.96.36.199 amd64 Neuron profile and debug tools ii tensorflow-model-server-neuron 188.8.131.52.4.0.0 all TensorFlow Serving ModelServer
Launch the pre-installed Tensorflow Neuron development environment on the Deep Learning AMI and run the update process.
source activate aws_neuron_tensorflow_p36 pip config set global.extra-index-url https://pip.repos.neuron.amazonaws.com pip install --upgrade tensorflow-neuron tensorboard-plugin-neuron neuron-cc
Check the installed Tensorflow Neuron development environment.
pip list | grep neuron
The latest version as of June 1, 2021 is as follows. You can see that the Neuron compiler, Tensorflow Neuron, etc. are installed.
neuron-cc 184.108.40.206+737cbb69a tensorboard-plugin-neuron 220.127.116.11 tensorflow-neuron 18.104.22.168.4.0.0
Software tools and packages are updated frequently, so it’s important to make sure you’re always up to date.
Starting with Neuron SDK 1.14.0, we will no longer support conda packages to install Neuron SDK framework in DLAMI. Please refer to blog.