Convolutional Neural Network

Hui Lin @Google

Types of Neural Network

Computer Vision

Image Data

Image Data

Convolutions

HTML5 Icon

Edge Detection

Parameters

Padding

Strided convolutions

Summary of Convolutions

Convolutions Over Volume

Your Turn: Number of Parameters in One Layer

Question: If you have 10 filters that are \(3 \times 3 \times 3\) in one layer of a neural network, how many parameters does that layer have?

Pooling Layers

Pooling Layers

Types of Layer in A Convolutional Network

Classic CNNs: LeNet-5

Classic CNNs: AlexNet

Classic CNNs: VGG-16

Classic CNNs: ResNets

Using Keras To Build CNN

Typical keras workflow:

  1. Define your training data: input tensors and target tensors
  2. Define a network of layers (or models) that maps your inputs to your targets
  3. Configure the learning process by choosing a loss function, an optimizer, and some metrics to monitor
  4. Iterate on your training data by calling the fit() method of your model

Using Keras To Build CNN

# Define model structure
cnn_model <- keras_model_sequential() %>%
  layer_conv_2d(filters = 32, kernel_size = c(3, 3), 
  activation = "relu", input_shape = input_shape) %>%
  layer_max_pooling_2d(pool_size = c(2, 2)) %>%
  layer_conv_2d(filters = 64, kernel_size = c(3, 3), activation = "relu") %>%
  layer_dropout(rate = 0.25) %>%
  layer_flatten() %>%
  layer_dense(units = 128, activation = "relu") %>%
  layer_dropout(rate = 0.5) %>%
  layer_dense(units = num_classes, activation = "softmax")

Using Keras To Build CNN

# Compile model
cnn_model %>% compile(
  loss = loss_categorical_crossentropy,
  optimizer = optimizer_adadelta(),
  metrics = c('accuracy')
)

Using Keras To Build CNN

# Train model
cnn_history <- cnn_model %>%
  fit(
    x_train, y_train,
    batch_size = batch_size,
    epochs = epochs,
    validation_split = 0.2
  )
# Model prediction
cnn_pred <- cnn_model %>%
  predict_classes(x_test)

Size of the Model

Different Architecture Search Algorithms:

Understanding Neural Networks