Running on CPU

Before experiencing accelerated high performance inference by Neuron cores, let’s run Resnet-50 model on the CPU first.

Inf1 instances include the latest 2nd generation Intel® Xeon® Scalable processors. inf1.2xlarge supports 8 vCPUs.

Step 1. Create a python script for inference

Create a python script named infer_resnet50_cpu.py with the following content:

import os
import time
import shutil
import numpy as np
import tensorflow as tf
import tensorflow.compat.v1.keras as keras
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.resnet50 import ResNet50, preprocess_input, decode_predictions

# Instantiate Keras ResNet50 model
keras.backend.set_learning_phase(0)
tf.keras.backend.set_image_data_format('channels_last')
model = ResNet50(weights='imagenet') 

# Export SavedModel
model_dir = 'resnet50'
shutil.rmtree(model_dir, ignore_errors=True)

tf.saved_model.simple_save(
    session            = keras.backend.get_session(),
    export_dir         = model_dir,
    inputs             = {'input': model.inputs[0]},
    outputs            = {'output': model.outputs[0]})

# Create input from image
img_sgl = image.load_img('kitten_small.jpg', target_size=(224, 224))
img_arr = image.img_to_array(img_sgl)
img_arr2 = np.expand_dims(img_arr, axis=0)
img_arr3 = preprocess_input(img_arr2)

# Run inference, Display results
preds = model.predict(img_arr3)
print(decode_predictions(preds, top=5)[0])

Step 2. Prepare input image data

Download the following sample image to perform image classification by Keras Resnet-50 model.

curl -O https://raw.githubusercontent.com/awslabs/mxnet-model-server/master/docs/images/kitten_small.jpg

Step 3. Run inference script

Run the inference script infer_resnet50_cpu.py for running inference on CPUs.

python infer_resnet50_cpu.py

You will get the following result.

[('n02123045', 'tabby', 0.68324643), ('n02127052', 'lynx', 0.12829497), ('n02123159', 'tiger_cat', 0.089623705), ('n02124075', 'Egyptian_cat', 0.06437764), ('n02128757', 'snow_leopard', 0.009918912)]