CNN : Lesson 1. Convolutional Neural Network (8)

27. Convolutional Layers (Part 2)

$\downarrow$ 해당 Lesson의 #27 Concept에서의 Jupyter Notebook

conv_visualization.ipynb

Convolutional Layer

In this notebook, we visualize four filtered outputs (a.k.a. activation maps) of a convolutional layer.

In this example, we are defining four filters that are applied to an input image by initializing the weights of a convolutional layer, but a trained CNN will learn the values of these weights.
Convolutional Layer의 weight = 필터(들)의 요소 값
처음에는 Random Initialization하나 학습을 통해 weight값 발전

Import the image

import cv2
import matplotlib.pyplot as plt

%matplotlib inline

img_path = 'springmeal.jpg`           # 로컬PC Jupyter경로에 춘식사진 받아놓음


# load color image
bgr_img = cv2.imread(img_path)

# convert to grayscale
gray_img = cv2.cvtColor(bgr_img, cv2.COLOR_BGR2GRAY)

# normalize, rescale entries to lie in 0~1
gray_img = gray_img.astype('float32')/255

# plot image
plt.imshow(gray_img, cmap='gray')
plt.show()

Define and visualize the filters

import numpy as np

filter_vals = np.array([[-1,-1,1,1], [-1,-1,1,1], [-1,-1,1,1], [-1,-1,1,1]])

filter_1 = filter_vals
filter_2 = -filter_1
filter_3 = filter_1.T
filter_4 = -filter_3
filters = np.array([filter_1, filter_2, filter_3, filter_4])

print(filters)
print(filters.shape)

>>>
array([[[-1, -1,  1,  1],
        [-1, -1,  1,  1],
        [-1, -1,  1,  1],
        [-1, -1,  1,  1]],

       [[ 1,  1, -1, -1],
        [ 1,  1, -1, -1],
        [ 1,  1, -1, -1],
        [ 1,  1, -1, -1]],

       [[-1, -1, -1, -1],
        [-1, -1, -1, -1],
        [ 1,  1,  1,  1],
        [ 1,  1,  1,  1]],

       [[ 1,  1,  1,  1],
        [ 1,  1,  1,  1],
        [-1, -1, -1, -1],
        [-1, -1, -1, -1]]])
        
(4, 4, 4)

위 출력결과와 같은 4x4x4 NumPy array를 filters에 할당

fig = plt.figure(figsize=(10, 5))
for i in range(4):
    ax = fig.add_subplot(1, 4, i+1, xticks=[], yticks=[])
    ax.imshow(filters[i], cmap='gray')
    ax.set_title('Filter %s' % str(i+1))
    width, height = filters[i].shape
    for x in range(width):
        for y in range(height):
            ax.annotate(str(filters[i][x][y]), xy=(y,x),
                        horizontalalignment='center',
                        verticalalignment='center',
                        color='white' if filters[i][x][y]<0 else 'black')

filters를 시각화해본 모습

Define a convolutional layer

For a convolutional neural network, we'll start by defining a:

• Convolutional layer

Initialize a single convolutional layer so that it contains all your created filters. Note that you are not training this network; you are initializing the weights in a convolutional layer so that you can visualize what happens after a forward pass through this network!

CNN을 위해서, Convolutional Layer를 먼저 만들어볼것이다.
[!] 지금 단계에서는 Convolutional Layer의 필터 weight값을 훈련시키는것이 아니다.

__init__ and forward

To define a neural network in PyTorch, you define the layers of a model in the function __init__ and define the forward behavior of a network that applies those initialized layers to an input (x) in the function forward. In PyTorch we convert all inputs into the Tensor datatype, which is similar to a list data type in Python.

Below, I define the structure of a class called Net that has a convolutional layer that can contain four 4x4 grayscale filters.
PyTorch에서의 (model의) Layer를 만들때에는 꼭? __init__과 forward method가 있어야한다.
아래 코드에서 gray이미지에 대한 4x4 filter를 가진 Convolutional Layer를 가진 Class를 만들어보겠다.

import torch
import torch.nn as nn
import torch.nn.functional as F
    
# define a neural network with a single convolutional layer with four filters
class Net(nn.Module):
    
    def __init__(self, weight):
        super(Net, self).__init__()
        # initializes the weights of the convolutional layer to be the weights of the 4 defined filters
        k_height, k_width = weight.shape[2:]
        # assumes there are 4 grayscale filters
        self.conv = nn.Conv2d(1, 4, kernel_size=(k_height, k_width), bias=False)
        self.conv.weight = torch.nn.Parameter(weight)

    def forward(self, x):
        # calculates the output of a convolutional layer
        # pre- and post-activation
        conv_x = self.conv(x)
        activated_x = F.relu(conv_x)
        
        # returns both layers
        return conv_x, activated_x
    
# instantiate the model and set the weights
weight = torch.from_numpy(filters).unsqueeze(1).type(torch.FloatTensor)
model = Net(weight)

# print out the layer in the network
print(model)

>>>
Net(
  (conv): Conv2d(1, 4, kernel_size=(4, 4), stride=(1, 1), bias=False)
)

• k_height, k_width = weight.shape[2:] : 이 코드보다 더 아래의 코드를 보면 weight가 torch.from_numpy(filters), 즉 아까 선언한 filters (4x4x4)로부터 온다는 것을 알 수 있다. 그건 그렇고 .shape[2:]는 뭘까?

[+] Tensor의 .squeeze : 1인 차원을 제거
[+] Tensor의 .unsqueeze : 0을 넣어주면 1차원, 1을 넣어주면 2차원, ... 위치에 1인 차원을 추가해준다.
[!] shape는 어떤 실제 tensor나 array를 반환하는게 아니라 진짜 shape에 대한 정보만 텍스트로 반환한다. (아래 코드를 봐보자)

# weight = torch.from_numpy(filters).unsqueeze(1).type(torch.FloatTensor)
weight.shape

>>>
torch.Size([4, 1, 4, 4])      # .unsqueeze(1)을 통해 4x4x4 tensor가 4x1x4x4 tensor가 됨

---------------------------------

a, b = weight.shape[2:]
print(a,b)

>>>
4 4

• self.conv = nn.Conv2d(1, 4, kernel_size=(k_height, k_width), bias=False) : PyTorch의 nn의 Conv2d 을 사용함.

# helper function for visualizing the output of a given layer
# default number of filters is 4
def viz_layer(layer, n_filters= 4):
    fig = plt.figure(figsize=(20, 20))
    
    for i in range(n_filters):
        ax = fig.add_subplot(1, n_filters, i+1, xticks=[], yticks=[])
        # grab layer outputs
        ax.imshow(np.squeeze(layer[0,i].data.numpy()), cmap='gray')
        ax.set_title('Output %s' % str(i+1))
        

[+] Layer로부터의 output image를 시각화해줄 helper function임 (참고만바람)

Let's look at the output of a convolutional layer, before and after a ReLu activation function is applied.
`convolutional layer

# plot original image
plt.imshow(gray_img, cmap='gray')

# visualize all filters
fig = plt.figure(figsize=(12, 6))
fig.subplots_adjust(left=0, right=1.5, bottom=0.8, top=1, hspace=0.05, wspace=0.05)
for i in range(4):
    ax = fig.add_subplot(1, 4, i+1, xticks=[], yticks=[])
    ax.imshow(filters[i], cmap='gray')
    ax.set_title('Filter %s' % str(i+1))

    
# convert the image into an input Tensor
gray_img_tensor = torch.from_numpy(gray_img).unsqueeze(0).unsqueeze(1)

# get the convolutional layer (pre and post activation)
conv_layer, activated_layer = model(gray_img_tensor)

# visualize the output of a conv layer
viz_layer(conv_layer)