Using VGG16 Pre-trained on ImageNet¶
CLICK HERE to see the repo
Using VGG16 pretrained on ImageNet for a new task by replacing the classifier at the top of the network¶
The jupyter notebook features in this repo shows how to use VGG16 (pretrained on ImageNet) for a new classification task. It involves using a new dataset and replacing the classifier (the fully connected layers at top of the network) with a new classifier. The base of the network (convolutional base) is frozen and the new classifier is trained using the new dataset.
In [7]:
import tensorflow.keras.backend as K
from tensorflow.keras.models import Model
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Input
from tensorflow.keras.layers import Flatten
from tensorflow.keras import optimizers
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from tensorflow.keras.preprocessing import image
from sklearn.metrics import classification_report
import numpy as np
import pickle
import matplotlib.pyplot as plt
from tensorflow.keras.applications.vgg16 import VGG16
In [8]:
K.clear_session()
dataset = "../data_set"
Set up Keras ImageDataGenerator:¶
- rotation_range adds a random rotation within the range of +/- the value.
- width_shift_range a random horizontal shift of the image within the range of +/- the value.
- height_shift_range a random vertical shift of the image within the range of +/- the value.
- shear_range a random stretch of the image at an angle within the range of +/- the value.
- horizontal_flip a allows random flips of the image about the horizontal axis
- fill_mode some augmentations create empty spaces, fill mode determines how we fill these spaces. "nearest" repeats the pixels and the edge of the space.
In [9]:
trdata = ImageDataGenerator(rescale= 1./255,
rotation_range=40,
width_shift_range=0.2,
height_shift_range=0.2,
shear_range = 0.2,
zoom_range=0.2,
horizontal_flip=True,
fill_mode='nearest')
Using the ImageDataGenerator to load data from the train, validation and test sets
In [10]:
traindata = trdata.flow_from_directory(directory=dataset+"/train",
target_size = (224,224),
color_mode = 'rgb',
class_mode = 'binary',
batch_size=20,
shuffle=True,
seed=42)
In [11]:
valdata = ImageDataGenerator(rescale= 1./255)
validatedata = valdata.flow_from_directory(directory=dataset+"/validate",
target_size = (224,224),
color_mode = 'rgb',
class_mode = 'binary',
batch_size=20)
In [30]:
tsdata = ImageDataGenerator(rescale= 1./255)
testdata = tsdata.flow_from_directory(directory=dataset+"/test",
target_size = (224,224),
color_mode = 'rgb',
class_mode = 'binary',
batch_size=20,
shuffle=False)
Loading VGG16 model with weights after being pre-trained on ImageNet¶
In [13]:
label_map = (traindata.class_indices)
print(label_map)
vggmodel = VGG16(weights="imagenet", include_top='False', input_shape =(224,224,3))
Creating a new model from VGG16 using the Keras Sequential model API¶
- Create blank model
- Add the all layers from VGG16 except the layers at the top of the network (the classifier)
- Ensure layers are frozen .i.e. not trainable
In [14]:
mymodel = Sequential()
for layer in vggmodel.layers[:-4]:
print(layer.name)
mymodel.add(layer)
del vggmodel
for layer in mymodel.layers:
print(layer.name, layer.trainable)
layer.trainable = False
Adding a classifier to the top of the new model¶
Adds on 2 fully-connected layers and the final output layer is a single neuron with a sigmoid activation function
This model is going to be used to perform binary classification so a sigmoid is used as the final layer.
The final output is a probability, so a range of 0-1 for the desired class.
In [15]:
new_top = Sequential()
new_top.add(Flatten())
new_top.add(Dense(1024, activation = 'relu'))
new_top.add(Dense(512, activation ='relu'))
new_top.add(Dense(1, activation ='sigmoid'))
In [16]:
for layer in new_top.layers:
print(layer)
In [17]:
mymodel.add(new_top)
mymodel.summary()
Configure model for training.¶
- optimizer the backpropagation optimization algorithm being used. Adam is typically used with deep neural networks
- loss is the loss function. For a binary classifcation function we use the binary_crossentropy loss function
- metrics the list of metrics to monitor during training
In [18]:
mymodel.compile(optimizers.Adam(lr=0.0001), loss='binary_crossentropy',
metrics =['accuracy'])
The fit_generator provides a way to configure additional training settings.
- generator is for the training data generator
- steps_per_epoch typically determined by the number of samples in the data set divided by the batch size for the training data generator
- validation_data is for the validation data generator
- validation_steps similar to steps_per_epoch, determined by the number of samples in the data set divided by the batch size for the validation data generator
- epochs The number of epoch to train the model for. An epoch is a single iteration over all of the training data
- verbose The level of detail about training information output during training
In [19]:
history = mymodel.fit_generator(generator=traindata, steps_per_epoch=137,
validation_data= validatedata, validation_steps=35,
epochs=15,
verbose=2)
Uses pickle (Python object serialization) to store training information as a binary file.
In [20]:
with open('vgg16/vgg16_p1_trainHistoryDict', 'wb') as file_pi:
pickle.dump(history.history, file_pi)
Plot the model accuracy and loss during training¶
Loss and accuracy graphs of the model on the training and validation sets are plotted.
In [21]:
plt.plot(history.history['accuracy'],linestyle='--', marker='o')
plt.plot(history.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
# summarize history for loss
plt.plot(history.history['loss'],linestyle='--', marker='o')
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
Generating a classification report¶
Shows how a classification report can be plotted using model training history.
Uses the scikit-learn classification_report() function
In [31]:
K.clear_session() #seems to help prevent a random memoryleak
y_pred = mymodel.predict(testdata, verbose=1)
y_bool =[]
for data in y_pred:
if data <0.5:
y_bool.append(0)
else:
y_bool.append(1)
In [32]:
report = classification_report(testdata.classes, y_bool, output_dict=False)
print(report)
In [ ]: