권적 신경 네트워크의 혼란: 그레이스케일에서 RGB로의 전환

CNN을 망치다

CNN은 지난 몇 년 동안 매우 유명하고 인기가 많았으며, 현재 많은 가장 선진적인 기술들이 이곳에서 컴퓨터 시각으로 놀라운 일을 하고 있다.이 연구들의 이름을 열거하는 것은 막을 수 없다.처음 몇 권의 노트에서 나는 그레이스케일에서 RGB로의 전환을 고려했다.나의 일은 완전하지 않을 수도 있지만, 나는 그것을 갱신하고 더 많은 개념을 추가할 계획이다.나는 단지 CNN을 망쳤을 뿐이다.

그레이스케일 이미지를 RGB로 변환

이 블로그에서 CNN을 만들어 그레이스케일 이미지를 입력하고 그에 상응하는 RGB 이미지와 비교하는 것 외에 시간이 지날수록 얼마나 잘 나올지 확인하는 것 외에 나는 놀라운 일을 하지 않았다.
RGB에서 회색조까지는 반전할 수 없는 과정으로 일단 완성되면 원시적인 색으로 착색할 수 없다는 것은 잘 알려져 있다.그러나 최근 몇 년 동안, 많은 가장 선진적인 기능들이 이것을 가능하게 했다.
이런 기술에 대해 나는 두 가지 방법을 시험해 보았다.

입력은 회색조, 출력은 RGB 이미지

실험실의 L 값으로 입력하고 실험실의 AB 값으로 출력한다.

LAB는 밝기(우리가 아는 것은 그레이스케일)를 나타내고 AB는 각각 빨간색/녹색과 파란색/노란색을 나타낸다.More Here .

초보 단계

가상 디스플레이 기능

단지 우리의 이미지를 큰 숫자에 보여주고 싶었을 뿐이다.

import os
import numpy as np
import cv2
import matplotlib.pyplot as plt

def show(img, fig_size=(10, 10)):
  figure = plt.figure(figsize=fig_size)
  plt.imshow(img)
  plt.xticks([])
  plt.yticks([])
  plt.show()

img = np.random.randint(0, 255, (100, 100))  
show(img)

데이터 세트 준비

이 문제에 대해 우리는 대량의 데이터 집합이 필요하다. 나는 Kaggle과 다른 출처에서 공개된 데이터 집합을 사용할 것이다.하드웨어 수요를 충족시키기 위해 Google colab을 사용합니다. 왜냐하면 저는 큰 저장 공간과RAM, 그리고 교육에 사용되는 GPU를 가지고 있기 때문입니다.나의 유일한 관심사는 모델 파일의 구조와 무게다.따라서 COLAB 커널에서 다운로드한 모든 이미지가 반드시 드라이브에 저장되는 것은 아닙니다.

데이터 세트: 고양이와 개

누가 이 데이터 집합을 기억하지 못합니까?

import zipfile
import os

!wget --no-check-certificate \
    https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip \
    -O /tmp/cats_and_dogs_filtered.zip
local_zip = '/tmp/cats_and_dogs_filtered.zip'
zip_ref = zipfile.ZipFile(local_zip, 'r')
zip_ref.extractall('/tmp')
zip_ref.close()


--2021-02-28 13:11:55-- https://storage.googleapis.com/mledu-datasets/cats_and_dogs_filtered.zip
Resolving storage.googleapis.com (storage.googleapis.com)... 74.125.197.128, 74.125.142.128, 74.125.195.128, ...
Connecting to storage.googleapis.com (storage.googleapis.com)|74.125.197.128|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 68606236 (65M) [application/zip]
Saving to: ‘/tmp/cats_and_dogs_filtered.zip’

/tmp/cats_and_dogs_ 100%[===================>] 65.43M 170MB/s in 0.4s    

2021-02-28 13:11:56 (170 MB/s) - ‘/tmp/cats_and_dogs_filtered.zip’ saved [68606236/68606236]


base_dir = '/tmp/cats_and_dogs_filtered'
train_dir = os.path.join(base_dir, 'train')
validation_dir = os.path.join(base_dir, 'validation')

# Directory with our training cat pictures
train_cats_dir = os.path.join(train_dir, 'cats')

# Directory with our training dog pictures
train_dogs_dir = os.path.join(train_dir, 'dogs')

# Directory with our validation cat pictures
validation_cats_dir = os.path.join(validation_dir, 'cats')

# Directory with our validation dog pictures
validation_dogs_dir = os.path.join(validation_dir, 'dogs')

train_cat_fnames = os.listdir(train_cats_dir)
print(train_cat_fnames[:10])

train_dog_fnames = os.listdir(train_dogs_dir)
train_dog_fnames.sort()
print(train_dog_fnames[:10])


['cat.922.jpg', 'cat.994.jpg', 'cat.965.jpg', 'cat.721.jpg', 'cat.856.jpg', 'cat.941.jpg', 'cat.419.jpg', 'cat.974.jpg', 'cat.466.jpg', 'cat.259.jpg']
['dog.0.jpg', 'dog.1.jpg', 'dog.10.jpg', 'dog.100.jpg', 'dog.101.jpg', 'dog.102.jpg', 'dog.103.jpg', 'dog.104.jpg', 'dog.105.jpg', 'dog.106.jpg']

마성이공대학 지점 데이터 집합

다양한 장면이 포함됩니다.

# mit places data http://places.csail.mit.edu/
!wget http://data.csail.mit.edu/places/places205/testSetPlaces205_resize.tar.gz
!tar -xzf testSetPlaces205_resize.tar.gz


--2021-02-28 13:12:05-- http://data.csail.mit.edu/places/places205/testSetPlaces205_resize.tar.gz
Resolving data.csail.mit.edu (data.csail.mit.edu)... 128.52.129.40
Connecting to data.csail.mit.edu (data.csail.mit.edu)|128.52.129.40|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2341250899 (2.2G) [application/octet-stream]
Saving to: ‘testSetPlaces205_resize.tar.gz’

testSetPlaces205_re 100%[===================>] 2.18G 15.9MB/s in 1m 46s  

2021-02-28 13:13:51 (21.1 MB/s) - ‘testSetPlaces205_resize.tar.gz’ saved [2341250899/2341250899]

Kaggle: API 키

COLAB나 어느 곳에서든kaggle 데이터를 사용할 수 있도록kaggle의 API 키가 필요합니다.일단 우리의 계좌가 허락되면 쉽게 다운로드할 수 있다.

# to get kaggle dataset
!pip install kaggle
# uplaod kaggle.json firsst
!mkdir -p ~/.kaggle
!cp kaggle.json ~/.kaggle/
!chmod 600 ~/.kaggle/kaggle.json


Requirement already satisfied: kaggle in /usr/local/lib/python3.7/dist-packages (1.5.10)
Requirement already satisfied: python-dateutil in /usr/local/lib/python3.7/dist-packages (from kaggle) (2.8.1)
Requirement already satisfied: python-slugify in /usr/local/lib/python3.7/dist-packages (from kaggle) (4.0.1)
Requirement already satisfied: certifi in /usr/local/lib/python3.7/dist-packages (from kaggle) (2020.12.5)
Requirement already satisfied: six>=1.10 in /usr/local/lib/python3.7/dist-packages (from kaggle) (1.15.0)
Requirement already satisfied: urllib3 in /usr/local/lib/python3.7/dist-packages (from kaggle) (1.24.3)
Requirement already satisfied: tqdm in /usr/local/lib/python3.7/dist-packages (from kaggle) (4.41.1)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from kaggle) (2.23.0)
Requirement already satisfied: text-unidecode>=1.3 in /usr/local/lib/python3.7/dist-packages (from python-slugify->kaggle) (1.3)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->kaggle) (3.0.4)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->kaggle) (2.10)

카그르: 데이터 세트

비결은 데이터 세트의 URL로 이동한 다음 뒤에 있는 문자를 복사하는 것이다.com.i, e 만약에 제 데이터 집합 URL이 https://www.kaggle.com/qramkrishna/corn-leaf-infection-dataset이라면 저는 qramkrishna/옥수수잎 감염 데이터 집합만 필요합니다.

화훼 데이터 세트
누가 꽃을 좋아하지 않습니까?파일이 데이터 세트와 동일한 이름의 ZIP로 다운로드됩니다.우리의 내부 핵 작업 구역에서 그것을 추출합니다.

!kaggle datasets download -d alxmamaev/flowers-recognition

local_zip = 'flowers-recognition.zip'
zip_ref = zipfile.ZipFile(local_zip, 'r')
zip_ref.extractall('flowers-recognition')
zip_ref.close()

fdir = "flowers-recognition/flowers"
flowers_dir = [os.path.join(fdir, fd) for fd in os.listdir(fdir)]
flowers_dir


Downloading flowers-recognition.zip to /content
 98% 441M/450M [00:04<00:00, 103MB/s] 
100% 450M/450M [00:04<00:00, 102MB/s]

['flowers-recognition/flowers/tulip',
 'flowers-recognition/flowers/sunflower',
 'flowers-recognition/flowers/daisy',
 'flowers-recognition/flowers/flowers',
 'flowers-recognition/flowers/rose',
 'flowers-recognition/flowers/dandelion']

과일 데이터 세트
맛있다!

!kaggle datasets download -d moltean/fruits

local_zip = 'fruits.zip'
zip_ref = zipfile.ZipFile(local_zip, 'r')
zip_ref.extractall('fruits')
zip_ref.close()


Downloading fruits.zip to /content
 98% 748M/760M [00:19<00:00, 54.9MB/s]
100% 760M/760M [00:19<00:00, 40.9MB/s]


ftrain = "/content/fruits/fruits-360/Training/"
ftrain_dir = [os.path.join(ftrain, fd) for fd in os.listdir(ftrain)]

ftest = "/content/fruits/fruits-360/Test/"
ftest_dir = [os.path.join(ftest, fd) for fd in os.listdir(ftest)]
ftest_dir

옥수수 감염 데이터 세트
내가 왜 이걸 선택했지?

!kaggle datasets download -d qramkrishna/corn-leaf-infection-dataset

local_zip = 'corn-leaf-infection-dataset.zip'
zip_ref = zipfile.ZipFile(local_zip, 'r')
zip_ref.extractall('corn-leaf-infection-dataset')
zip_ref.close()


Downloading corn-leaf-infection-dataset.zip to /content
100% 13.0G/13.0G [05:46<00:00, 39.6MB/s]
100% 13.0G/13.0G [05:46<00:00, 40.2MB/s]


ctrain = ["/content/corn-leaf-infection-dataset/Corn Disease detection/Healthy corn", 
          "/content/corn-leaf-infection-dataset/Corn Disease detection/Infected"]

Inria 데이터 세트

!wget ftp://ftp.inrialpes.fr/pub/lear/douze/data/INRIAPerson.tar
!tar -xvf INRIAPerson.tar
!rm INRIAPerson.tar


inria_train = ['/content/INRIAPerson/70X134H96/Test/pos',
               "/content/INRIAPerson/96X160H96/Train/pos",
               "/content/INRIAPerson/train_64x128_H96/neg",
               "/content/INRIAPerson/train_64x128_H96/pos",
               "/content/INRIAPerson/Train/neg", 
               "/content/INRIAPerson/Train/pos"]
inria_test = ["/content/INRIAPerson/test_64x128_H96/pos",
              "/content/INRIAPerson/test_64x128_H96/neg",
              "/content/INRIAPerson/Test/neg", 
              "/content/INRIAPerson/Test/pos"]


# delete these huge zip file to prevent memory full
!rm corn-leaf-infection-dataset.zip
!rm testSetPlaces205_resize.tar.gz

입력은 회색조, 출력은 RGB 이미지

이것은 전통적인 인코더로 기존 층에 새로운 2층을 추가하려고 시도했지만, 내가 생각했던 것처럼 그렇게 간단하지는 않았다.

사용자 정의 데이터 생성기

가능한 이미지의 루트 디렉터리를 가져오고 모든 이미지의 전체 경로를 목록에 저장합니다.그리고 카드를 섞어 무작위 효과를 낸다.이 두 가지 경우 모두 같은 생성기를 사용할 수 있으며, 우리는 그것에 대해 소량의 편집만 할 수 있다.

from tensorflow.keras.utils import Sequence

img_size = (224, 224)
class ImageGenerator(Sequence):
  def __init__ (self, dirs=[], 
              target_size=(224,224), batch_size=32):
    self.batch_size=batch_size
    self.target_size = target_size
    self.dirs = dirs
    self.all_dirs = []
    for dir in self.dirs:
      self.all_dirs.extend([os.path.join(dir, fname) for fname in os.listdir(dir)])

    self.x = np.arange(len(self.all_dirs))
    np.random.shuffle(self.x)

  def __len__ (self):
        return int(np.ceil(len(self.x)/float(self.batch_size)))

  def __getitem__ (self, idx):
      batch_x = self.x[idx * self.batch_size:(idx+1) * self.batch_size]
      #batch_y = self.y[idx * self.batch_size:(idx+1) * self.batch_size]

      x, y = self.generate_image(batch_x)
      #print(batch.shape)
      return x, y

  def generate_image(self, ids):
    image_size = self.target_size
    batch_x = []
    batch_y = []
    for i in ids:
      try:
        bgr = cv2.imread(self.all_dirs[i], 1)
        bgr = cv2.resize(bgr, image_size)

        rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)
        gray = cv2.cvtColor(bgr, cv2.COLOR_BGR2GRAY)
        batch_x.append(gray)
        batch_y.append(rgb)
      except:
        pass
    batch_x = np.array(batch_x).reshape(len(batch_x), image_size[0], image_size[1], 1)/255
    batch_y = np.array(batch_y).reshape(len(batch_y), image_size[0], image_size[1], 3)/255

    return batch_x, batch_y

tdirs=['/tmp/cats_and_dogs_filtered/train/dogs',
      '/tmp/cats_and_dogs_filtered/train/cats',
       "/content/testSet_resize",
       ]

tdirs.extend(flowers_dir)
# tdirs.extend(ctrain)
tdirs.extend(inria_train)
# tdirs.extend(ftrain_dir)

vdirs = ['/tmp/cats_and_dogs_filtered/validation/dogs',
        '/tmp/cats_and_dogs_filtered/validation/cats']
vdirs.extend(inria_test)

# vdirs.extend(ftest_dir)

train_generator = ImageGenerator(tdirs, 
                           target_size=img_size, batch_size=64)

valid_generator = ImageGenerator(dirs=vdirs, 
                           target_size=img_size, batch_size=32)

for i in range(2):
  # print(generator. __getitem__ (i)[1][0].shape)
  show(train_generator. __getitem__ (i)[0][0].reshape(img_size))
  show(train_generator. __getitem__ (i)[1][0].reshape(img_size[0], img_size[1], 3))

Keras Datagenerator는 사용자 정의 데이터 생성기를 작성하여 작업을 수행할 수 있도록 합니다.

우리는 방금 모든 그림의 루트를 제시한 다음에 그림의 이름을 읽고 모든 그림 경로가 있는 위치를 보여 줍니다.그리고 카드를 씻어요.

각 역원의 목록을 사용하고 인덱스로 횟수를 생성합니다.

RGB 형식의 이미지를 읽고 회색조로 변환했습니다.

이미지를 표준화하고 일괄 처리를 반환합니다.

모델 생성하기

나는 종종 다른 모델보다 더 많은 특성을 배웠기 때문에 예비 훈련 모델을 사용할 것이다.나는 필터를 사용하여 모델의 출력 모양을 탭 모양과 일치하게 했다.만약 우리가 다른 필터를 사용하려고 한다면, Resize 층을 사용하는 것이 가장 좋다.

from keras.applications.inception_resnet_v2 import preprocess_input, InceptionResNetV2
from keras.layers import Input, GlobalAveragePooling2D, Dense, Dropout, Lambda, Reshape, BatchNormalization, Conv2D, MaxPooling2D, UpSampling2D
from keras.models import Model
import tensorflow as tf

from keras.applications.inception_v3 import InceptionV3
from tensorflow.keras import layers
from tensorflow.keras import activations

def create_base_network(input_shape, output_shape):
    """Get the base network to do the feature extract for the latent embedding

    Args:
        input_shape (tuple): Shape of image tensor input

    Returns:
        keras.models.Model
    """
    # input = Input(shape=input_shape)
    img_input = Input(shape=input_shape, name = 'grayscale_input_layer')
    x = Conv2D(3, (3,3), padding= 'same', name = 'grayscale_RGB_layer', activation="relu")(img_input)
    # x=Lambda(lambda v: tf.cast(tf.compat.v1.spectral.fft(tf.cast(v,dtype=tf.complex64)),tf.float32))(x)
    # inception = InceptionResNetV2(input_shape = (input_shape[0], input_shape[1], 3), include_top = False, weights = 'imagenet')
    inception = InceptionV3(weights='imagenet',include_top=False,input_shape=(input_shape[0], input_shape[1], 3))
    inception.layers.pop() # Remove classification layer
    # inception.summary()
    inception = inception(x)

    inception = Conv2D(128, (3,3), padding= 'same')(inception)
    inception = BatchNormalization()(inception)
    inception = activations.relu(inception)
    upsample = UpSampling2D(2)(inception)

    upsample = Conv2D(128, (3,3))(upsample)
    upsample = BatchNormalization()(upsample)
    upsample = activations.relu(upsample)
    upsample = UpSampling2D(2)(upsample)

    upsample = Conv2D(64, (3,3))(upsample)
    upsample = BatchNormalization()(upsample)
    upsample = activations.relu(upsample)
    upsample = UpSampling2D(2)(upsample)

    upsample = Conv2D(64, (3,3), padding="same")(upsample)
    upsample = BatchNormalization()(upsample)
    upsample = activations.relu(upsample)
    upsample = UpSampling2D(2)(upsample)

    upsample = Conv2D(32, (3,3), padding="same")(upsample)
    upsample = BatchNormalization()(upsample)
    upsample = activations.relu(upsample)
    upsample = UpSampling2D(2)(upsample)

    upsample = Conv2D(2, (3,3), padding="same")(upsample)
    upsample = BatchNormalization()(upsample)
    upsample = activations.tanh(upsample)
    upsample = UpSampling2D(2)(upsample)

    upsample = Conv2D(3, (3,3), padding="same")(upsample)
    upsample = activations.sigmoid(upsample)

    # resize = Lambda(resize_layer, output_shape=resize_layer_shape, arguments={"shape":output_shape})(upsample)
    # this is resizing layer
    # resize = tf.keras.layers.experimental.preprocessing.Resizing(input_shape[0], input_shape[1])(upsample)

    # output = resize
    output=upsample
    model = Model(inputs=[img_input], outputs=[output])
    model.summary()
    return model
model = create_base_network((img_size[0],img_size[1], 1), (img_size[0], img_size[1], 3))


Model: "model_1"
_________________________________________________________________
Layer (type) Output Shape Param #   
=================================================================
grayscale_input_layer (Input [(None, 224, 224, 1)] 0         
_________________________________________________________________
grayscale_RGB_layer (Conv2D) (None, 224, 224, 3) 30        
_________________________________________________________________
inception_v3 (Functional) (None, 5, 5, 2048) 21802784  
_________________________________________________________________
conv2d_195 (Conv2D) (None, 5, 5, 128) 2359424   
_________________________________________________________________
batch_normalization_194 (Bat (None, 5, 5, 128) 512       
_________________________________________________________________
tf.nn.relu_5 (TFOpLambda) (None, 5, 5, 128) 0         
_________________________________________________________________
up_sampling2d_6 (UpSampling2 (None, 10, 10, 128) 0         
_________________________________________________________________
conv2d_196 (Conv2D) (None, 8, 8, 128) 147584    
_________________________________________________________________
batch_normalization_195 (Bat (None, 8, 8, 128) 512       
_________________________________________________________________
tf.nn.relu_6 (TFOpLambda) (None, 8, 8, 128) 0         
_________________________________________________________________
up_sampling2d_7 (UpSampling2 (None, 16, 16, 128) 0         
_________________________________________________________________
conv2d_197 (Conv2D) (None, 14, 14, 64) 73792     
_________________________________________________________________
batch_normalization_196 (Bat (None, 14, 14, 64) 256       
_________________________________________________________________
tf.nn.relu_7 (TFOpLambda) (None, 14, 14, 64) 0         
_________________________________________________________________
up_sampling2d_8 (UpSampling2 (None, 28, 28, 64) 0         
_________________________________________________________________
conv2d_198 (Conv2D) (None, 28, 28, 64) 36928     
_________________________________________________________________
batch_normalization_197 (Bat (None, 28, 28, 64) 256       
_________________________________________________________________
tf.nn.relu_8 (TFOpLambda) (None, 28, 28, 64) 0         
_________________________________________________________________
up_sampling2d_9 (UpSampling2 (None, 56, 56, 64) 0         
_________________________________________________________________
conv2d_199 (Conv2D) (None, 56, 56, 32) 18464     
_________________________________________________________________
batch_normalization_198 (Bat (None, 56, 56, 32) 128       
_________________________________________________________________
tf.nn.relu_9 (TFOpLambda) (None, 56, 56, 32) 0         
_________________________________________________________________
up_sampling2d_10 (UpSampling (None, 112, 112, 32) 0         
_________________________________________________________________
conv2d_200 (Conv2D) (None, 112, 112, 2) 578       
_________________________________________________________________
batch_normalization_199 (Bat (None, 112, 112, 2) 8         
_________________________________________________________________
tf.math.tanh_1 (TFOpLambda) (None, 112, 112, 2) 0         
_________________________________________________________________
up_sampling2d_11 (UpSampling (None, 224, 224, 2) 0         
_________________________________________________________________
conv2d_201 (Conv2D) (None, 224, 224, 3) 57        
_________________________________________________________________
tf.math.sigmoid_1 (TFOpLambd (None, 224, 224, 3) 0         
=================================================================
Total params: 24,441,313
Trainable params: 24,406,045
Non-trainable params: 35,268
_________________________________________________________________

훈련시키다

모델을 컴파일하기 위해 최적화기와 손실 함수를 정의합시다.

from keras.optimizers import Adam
adam = Adam(lr=0.001)
model.compile(optimizer=adam, loss='mse')

그 밖에 리셋을 사용하면 우리 모델이 정확하게 범화되는 데 도움이 될 것이다.EarlyStopping 과도한 조립이 발생할 때 모형 훈련을 정지할 수 있습니다.ReduceLROnPlateau 주어진 인자에 따라 학습률을 낮추어 최소치에 도달할 때까지 사용한다.우리는 또한 모든 역원에 있는 모델의 무게를 저장할 것이다.

from keras.callbacks import EarlyStopping, ModelCheckpoint, ReduceLROnPlateau, Callback

class ShowImagesOnEpochEnd(Callback):
    """ 
    Inherit from keras.callbacks.Callback
    """
    def __init__ (self, data=None, img_size=(224, 224)):
        """ 
        data: dataset to view from
        img_size: image size
        """
        self.data = data
        self.img_size = img_size

    def lab2RGB(self, X, Y):
      canvas = np.zeros((img_size[0], img_size[1], 3), dtype=np.float64)
      canvas[:,:,0] = X[0][:,:,0]
      canvas[:,:,1:] = Y[0]*128

      return lab2rgb(canvas)

    def on_epoch_end(self, epoch, logs={}):
        inds = np.random.randint(0, 20, 3)
        for i in inds:
          # print(logs)
          img=self.data. __getitem__ (i)[0][0].reshape(1, self.img_size[0], self.img_size[1], 1)

          gimg=self.data. __getitem__ (i)[0][0].reshape(1, self.img_size[0], self.img_size[1], 1)
          rgbimg = self.data. __getitem__ (i)[1][0].reshape(self.img_size[0], self.img_size[1], 3)


          plt.figure(figsize=(20,20))
          plt.subplot(1,2,1)
          plt.imshow(rgbimg)
          plt.title("True RGB Image")
          plt.xticks([])
          plt.yticks([])

          out = self.model.predict(img).reshape(self.img_size[0], self.img_size[1], 3)
          # out = self.lab2RGB(img, out)
          plt.subplot(1,2,2)
          plt.imshow(out)
          plt.title("Output RGB Image")
          plt.xticks([])
          plt.yticks([])
          plt.show()

callbacks = [
    EarlyStopping(patience=5, verbose=1),
    ReduceLROnPlateau(factor=0.9, patience=5, min_lr=0.0000001, verbose=1),
    ModelCheckpoint("/content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_{epoch:02d}.h5", verbose=1, save_weights_only=True),
    ShowImagesOnEpochEnd(data=valid_generator)
]

사용자 정의 리셋ShowImagesOnEpochEnd은 역원이 끝날 때의 그레이스케일 이미지 결과를 보여 줍니다.

history = model.fit(train_generator, 
          epochs=30,
          validation_data=valid_generator,
          callbacks=callbacks)


Epoch 1/30
881/881 [==============================] - 324s 350ms/step - loss: 0.0196 - val_loss: 0.0080    
Epoch 00001: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_01.h5

Epoch 2/30
881/881 [==============================] - 302s 343ms/step - loss: 0.0087 - val_loss: 0.0079

Epoch 00002: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_02.h5

Epoch 3/30
881/881 [==============================] - 304s 344ms/step - loss: 0.0085 - val_loss: 0.0078

Epoch 00003: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_03.h5

Epoch 4/30
881/881 [==============================] - 309s 350ms/step - loss: 0.0084 - val_loss: 0.0077

Epoch 00004: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_04.h5

Epoch 5/30
881/881 [==============================] - 315s 357ms/step - loss: 0.0083 - val_loss: 0.0079

Epoch 00005: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_05.h5

Epoch 6/30
881/881 [==============================] - 330s 374ms/step - loss: 0.0082 - val_loss: 0.0079

Epoch 00006: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_06.h5

Epoch 7/30
881/881 [==============================] - 342s 388ms/step - loss: 0.0083 - val_loss: 0.0078

Epoch 00007: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_07.h5

Epoch 8/30
881/881 [==============================] - 356s 404ms/step - loss: 0.0081 - val_loss: 0.0076

Epoch 00008: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_08.h5

Epoch 9/30
881/881 [==============================] - 368s 417ms/step - loss: 0.0081 - val_loss: 0.0077

Epoch 00009: ReduceLROnPlateau reducing learning rate to 0.0009000000427477062.

Epoch 00009: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_09.h5

Epoch 10/30
881/881 [==============================] - 383s 435ms/step - loss: 0.0080 - val_loss: 0.0075

Epoch 00010: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_10.h5

Epoch 11/30
881/881 [==============================] - 396s 449ms/step - loss: 0.0080 - val_loss: 0.0078

Epoch 00011: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_11.h5

Epoch 12/30
881/881 [==============================] - 408s 462ms/step - loss: 0.0079 - val_loss: 0.0075

Epoch 00012: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_12.h5

Epoch 13/30
881/881 [==============================] - 415s 471ms/step - loss: 0.0079 - val_loss: 0.0076

Epoch 00013: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_13.h5

Epoch 14/30
881/881 [==============================] - 424s 481ms/step - loss: 0.0078 - val_loss: 0.0075

Epoch 00014: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_14.h5

Epoch 15/30
881/881 [==============================] - 439s 498ms/step - loss: 0.0078 - val_loss: 0.0076

Epoch 00015: ReduceLROnPlateau reducing learning rate to 0.0008100000384729356.

Epoch 00015: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_15.h5

Epoch 16/30
881/881 [==============================] - 451s 512ms/step - loss: 0.0078 - val_loss: 0.0075

Epoch 00016: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_16.h5

Epoch 17/30
784/881 [=========================>....] - ETA: 46s - loss: 0.0077

나의 네트워크가 끊겼기 때문에 교육은 이것으로 끝났지만, 우리는 이 기술이 실망스럽다는 것을 분명히 볼 수 있다.그러나 우리가 방대한 데이터 집합을 보유한 후에 이것이 작용할 수도 있다.

L로 입력하고 랩 AB로 내보내기

나는 this 아주 좋은 블로그를 읽고 이 글을 썼다.발전기와 모델을 제외한 모든 부품은 상술한 것과 같다.

이미지 생성기

나는 skimage.color에서 RGB를 LAB와 Reverse로 바꾸는 두 가지 방법을 가져왔다.

from tensorflow.keras.utils import Sequence
from skimage.color import rgb2lab, lab2rgb

img_size = (224, 224)
class ImageGenerator(Sequence):
  def __init__ (self, dirs=[], 
              target_size=(224,224), batch_size=32):
    self.batch_size=batch_size
    self.target_size = target_size
    self.dirs = dirs
    self.all_dirs = []
    for dir in self.dirs:
      self.all_dirs.extend([os.path.join(dir, fname) for fname in os.listdir(dir)])

    self.x = np.arange(len(self.all_dirs))
    np.random.shuffle(self.x)

  def __len__ (self):
        return int(np.ceil(len(self.x)/float(self.batch_size)))

  def __getitem__ (self, idx):
      batch_x = self.x[idx * self.batch_size:(idx+1) * self.batch_size]
      #batch_y = self.y[idx * self.batch_size:(idx+1) * self.batch_size]

      x, y = self.generate_image(batch_x)
      #print(batch.shape)
      return x, y

  def generate_image(self, ids):
    image_size = self.target_size
    batch_x = []
    batch_y = []
    for i in ids:
      try:
        bgr = cv2.imread(self.all_dirs[i], 1)
        bgr = cv2.resize(bgr, image_size)
        image = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)

        X = rgb2lab(1.0/255*image)[:,:,0]
        Y = rgb2lab(1.0/255*image)[:,:,1:]
        Y = Y / 128
        X = X.reshape(1, image_size[0], image_size[1], 1)
        Y = Y.reshape(1, image_size[0], image_size[1], 2)

        batch_x.append(X)
        batch_y.append(Y)

        # rgb = cv2.cvtColor(bgr, cv2.COLOR_BGR2RGB)
        # gray = cv2.cvtColor(bgr, cv2.COLOR_BGR2GRAY)
        # batch_x.append(gray)
        # batch_y.append(rgb)
      except:
        pass
    # batch_x = np.array(batch_x).reshape(len(batch_x), image_size[0], image_size[1], 1)/255
    # batch_y = np.array(batch_y).reshape(len(batch_y), image_size[0], image_size[1], 3)/255
    batch_x = np.array(batch_x).reshape(len(batch_x), image_size[0], image_size[1], 1)
    batch_y = np.array(batch_y).reshape(len(batch_y), image_size[0], image_size[1], 2)

    return batch_x, batch_y

# tdirs =["/content/flowers-recognition/flowers/daisy"]

tdirs=['/tmp/cats_and_dogs_filtered/train/dogs',
      '/tmp/cats_and_dogs_filtered/train/cats',
       "/content/testSet_resize",
       ]

tdirs.extend(flowers_dir)
# tdirs.extend(ctrain)
tdirs.extend(inria_train)
# tdirs.extend(ftrain_dir)

# vdirs=["/content/flowers-recognition/flowers/daisy"]
vdirs = ['/tmp/cats_and_dogs_filtered/validation/dogs',
        '/tmp/cats_and_dogs_filtered/validation/cats']

# vdirs.extend(ftest_dir)
vdirs.extend(inria_test)

train_generator = ImageGenerator(tdirs, 
                           target_size=img_size, batch_size=64)

valid_generator = ImageGenerator(dirs=vdirs, 
                           target_size=img_size, batch_size=32)

def lab2RGB(X, Y):
  canvas = np.zeros((img_size[0], img_size[1], 3), dtype=np.float64)
  canvas[:,:,0] = X[0][:,:,0]
  canvas[:,:,1:] = Y[0]*128

  return lab2rgb(canvas)

for i in range(2):
  X = train_generator. __getitem__ (i)[0][0].reshape(1, img_size[0], img_size[1], 1)
  Y = train_generator. __getitem__ (i)[1][0].reshape(1, img_size[0], img_size[1], 2)

  # output = model.predict(X)
  show(lab2RGB(X, Y))
  show(X.reshape(img_size))

실험실의 최대치는 128이기 때문에 우리는 이 값을 제외하고 귀일화한다.

모델 생성하기

def create_base_network(input_shape, output_shape):
    img_input = Input(shape=input_shape, name = 'grayscale_input_layer')
    x = Conv2D(3, (3,3), padding= 'same', name = 'grayscale_RGB_layer', activation="relu")(img_input)
    # x=Lambda(lambda v: tf.cast(tf.compat.v1.spectral.fft(tf.cast(v,dtype=tf.complex64)),tf.float32))(x)
    # inception = InceptionResNetV2(input_shape = (input_shape[0], input_shape[1], 3), include_top = False, weights = 'imagenet')
    inception = InceptionV3(weights='imagenet',include_top=False,input_shape=(input_shape[0], input_shape[1], 3))
    inception.layers.pop() # Remove classification layer
    # inception.summary()
    for layer in inception.layers:
      layer.trainnable=False
    inception = inception(x)

    inception = Conv2D(128, (3,3), padding= 'same')(inception)
    inception = BatchNormalization()(inception)
    inception = activations.relu(inception)
    upsample = UpSampling2D(2)(inception)

    upsample = Conv2D(128, (3,3))(upsample)
    upsample = BatchNormalization()(upsample)
    upsample = activations.relu(upsample)
    upsample = UpSampling2D(2)(upsample)

    upsample = Conv2D(64, (3,3))(upsample)
    upsample = BatchNormalization()(upsample)
    upsample = activations.relu(upsample)
    upsample = UpSampling2D(2)(upsample)

    upsample = Conv2D(64, (3,3), padding="same")(upsample)
    upsample = BatchNormalization()(upsample)
    upsample = activations.relu(upsample)
    upsample = UpSampling2D(2)(upsample)

    upsample = Conv2D(32, (3,3), padding="same")(upsample)
    upsample = BatchNormalization()(upsample)
    upsample = activations.relu(upsample)
    upsample = UpSampling2D(2)(upsample)

    upsample = Conv2D(2, (3,3), padding="same")(upsample)
    upsample = BatchNormalization()(upsample)
    upsample = activations.tanh(upsample)
    upsample = UpSampling2D(2)(upsample)

    # upsample = Conv2D(3, (3,3), padding="same")(upsample)
    # upsample = activations.relu(upsample)

    # resize = Lambda(resize_layer, output_shape=resize_layer_shape, arguments={"shape":output_shape})(upsample)
    # this is resizing layer
    # resize = tf.keras.layers.experimental.preprocessing.Resizing(input_shape[0], input_shape[1])(upsample)

    # output = resize
    output=upsample
    model = Model(inputs=[img_input], outputs=[output])
    model.summary()
    return model
model = create_base_network((img_size[0],img_size[1], 1), (img_size[0], img_size[1], 3))


Model: "model_1"
_________________________________________________________________
Layer (type) Output Shape Param #   
=================================================================
grayscale_input_layer (Input [(None, 224, 224, 1)] 0         
_________________________________________________________________
grayscale_RGB_layer (Conv2D) (None, 224, 224, 3) 30        
_________________________________________________________________
inception_v3 (Functional) (None, 5, 5, 2048) 21802784  
_________________________________________________________________
conv2d_194 (Conv2D) (None, 5, 5, 128) 2359424   
_________________________________________________________________
batch_normalization_194 (Bat (None, 5, 5, 128) 512       
_________________________________________________________________
tf.nn.relu_5 (TFOpLambda) (None, 5, 5, 128) 0         
_________________________________________________________________
up_sampling2d_6 (UpSampling2 (None, 10, 10, 128) 0         
_________________________________________________________________
conv2d_195 (Conv2D) (None, 8, 8, 128) 147584    
_________________________________________________________________
batch_normalization_195 (Bat (None, 8, 8, 128) 512       
_________________________________________________________________
tf.nn.relu_6 (TFOpLambda) (None, 8, 8, 128) 0         
_________________________________________________________________
up_sampling2d_7 (UpSampling2 (None, 16, 16, 128) 0         
_________________________________________________________________
conv2d_196 (Conv2D) (None, 14, 14, 64) 73792     
_________________________________________________________________
batch_normalization_196 (Bat (None, 14, 14, 64) 256       
_________________________________________________________________
tf.nn.relu_7 (TFOpLambda) (None, 14, 14, 64) 0         
_________________________________________________________________
up_sampling2d_8 (UpSampling2 (None, 28, 28, 64) 0         
_________________________________________________________________
conv2d_197 (Conv2D) (None, 28, 28, 64) 36928     
_________________________________________________________________
batch_normalization_197 (Bat (None, 28, 28, 64) 256       
_________________________________________________________________
tf.nn.relu_8 (TFOpLambda) (None, 28, 28, 64) 0         
_________________________________________________________________
up_sampling2d_9 (UpSampling2 (None, 56, 56, 64) 0         
_________________________________________________________________
conv2d_198 (Conv2D) (None, 56, 56, 32) 18464     
_________________________________________________________________
batch_normalization_198 (Bat (None, 56, 56, 32) 128       
_________________________________________________________________
tf.nn.relu_9 (TFOpLambda) (None, 56, 56, 32) 0         
_________________________________________________________________
up_sampling2d_10 (UpSampling (None, 112, 112, 32) 0         
_________________________________________________________________
conv2d_199 (Conv2D) (None, 112, 112, 2) 578       
_________________________________________________________________
batch_normalization_199 (Bat (None, 112, 112, 2) 8         
_________________________________________________________________
tf.math.tanh_1 (TFOpLambda) (None, 112, 112, 2) 0         
_________________________________________________________________
up_sampling2d_11 (UpSampling (None, 224, 224, 2) 0         
=================================================================
Total params: 24,441,256
Trainable params: 24,405,988
Non-trainable params: 35,268
_________________________________________________________________

훈련시키다

집성

from keras.optimizers import Adam
adam = Adam(lr=0.001)
model.compile(optimizer=adam, loss='mse')

답전

위에서 설명한 바와 같이 RGB까지 실험실을 사용할 것입니다.


from keras.callbacks import EarlyStopping, ModelCheckpoint, ReduceLROnPlateau, Callback

class ShowImagesOnEpochEnd(Callback):
    """ 
    Inherit from keras.callbacks.Callback
    """
    def __init__ (self, data=None, img_size=(224, 224)):
        """ 
        data: dataset to view from
        img_size: image size
        """
        self.data = data
        self.img_size = img_size

    def lab2RGB(self, X, Y):
      canvas = np.zeros((img_size[0], img_size[1], 3), dtype=np.float64)
      canvas[:,:,0] = X[0][:,:,0]
      canvas[:,:,1:] = Y[0]*128

      return lab2rgb(canvas)

    def on_epoch_end(self, epoch, logs={}):
        inds = np.random.randint(0, 20, 3)
        for i in inds:
          # print(logs)
          gimg=self.data. __getitem__ (i)[0][0].reshape(1, self.img_size[0], self.img_size[1], 1)
          abimg = self.data. __getitem__ (i)[1][0].reshape(1, self.img_size[0], self.img_size[1], 2)

          inp_img = self.lab2RGB(gimg, abimg)

          plt.figure(figsize=(20,20))
          plt.subplot(1,2,1)
          plt.imshow(inp_img.reshape(self.img_size[0], self.img_size[1], 3), cmap="gray")
          plt.title("Ture RGB Image")
          plt.xticks([])
          plt.yticks([])

          out = self.model.predict(gimg)
          output = self.lab2RGB(gimg, out)
          plt.subplot(1,2,2)
          plt.imshow(output)
          plt.title("Predicted RGB Image")
          plt.xticks([])
          plt.yticks([])
          plt.show()

callbacks = [
    EarlyStopping(patience=5, verbose=1),
    ReduceLROnPlateau(factor=0.9, patience=5, min_lr=0.0000001, verbose=1),
    ModelCheckpoint("/content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_LAB_{epoch:02d}.h5", verbose=1, save_weights_only=True),
    ShowImagesOnEpochEnd(data=valid_generator)
    ]

# model.load_weights("/content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_LAB_02.h5")


history = model.fit(train_generator, 
          epochs=30,
          validation_data=valid_generator,
          callbacks=callbacks)


Epoch 1/30
881/881 [==============================] - 2151s 2s/step - loss: 0.0212 - val_loss: 0.0075

Epoch 00001: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_LAB_01.h5

Epoch 2/30
881/881 [==============================] - 2133s 2s/step - loss: 0.0117 - val_loss: 0.0080

Epoch 00002: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_LAB_02.h5

Epoch 3/30
881/881 [==============================] - 2137s 2s/step - loss: 0.0108 - val_loss: 0.2515

Epoch 00003: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_LAB_03.h5

Epoch 4/30
881/881 [==============================] - 2150s 2s/step - loss: 0.0109 - val_loss: 0.0069

Epoch 00004: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_LAB_04.h5

Epoch 5/30
881/881 [==============================] - 2174s 2s/step - loss: 0.0114 - val_loss: 0.0134

Epoch 00005: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_LAB_05.h5

Epoch 6/30
881/881 [==============================] - 2196s 2s/step - loss: 0.0111 - val_loss: 0.0276

Epoch 00006: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_LAB_06.h5

Epoch 7/30
881/881 [==============================] - 2197s 2s/step - loss: 0.0111 - val_loss: 0.0064

Epoch 00007: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_LAB_07.h5

Epoch 8/30
881/881 [==============================] - 2202s 2s/step - loss: 0.0110 - val_loss: 0.0067

Epoch 00008: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_LAB_08.h5

Epoch 9/30
881/881 [==============================] - 2219s 3s/step - loss: 0.0112 - val_loss: 0.0072

Epoch 00009: saving model to /content/drive/MyDrive/Messing Up With CNN/GRAY2RGB_LAB_09.h5

Epoch 10/30
531/881 [=================>............] - ETA: 13:53 - loss: 0.0108

결론

그레이스케일을 RGB로 바꾸는 것은 점진적이지 않은 것 같다. 우리는 이미지의 전체 픽셀을 흐트러뜨리기 때문에 가치 있는 특징을 찾은 후에 착색이 더욱 어렵다.

LAB를 사용하는 것이 더 희망적인 것 같습니다. 왜냐하면 우리는 이미지의 AB값만 조정하고 원시 그레이스케일 이미지는 변하지 않기 때문입니다.또한 주의해야 할 것은 그레이스케일 이미지만 착색하는 것이다.

이미지를 컬러화하는 것은 대량의 이미지가 필요하기 때문에 어려운 작업이다.만약 우리가 많은 사람들의 모형을 훈련시키고 자연 이미지에 착색을 시도한다면, 그것은 틀림없이 실패할 것이다.

사상

만약 우리가 이러한 데이터 집합을 사용하지 않고 어떤 영화의 프레임 수 천 프레임을 읽고 그것을 사용하여 훈련을 한다면 우리는 데이터의 수요에서 벗어나 더 좋은 훈련 결과를 얻을 수 있을 것이다.

도구책

How to colorize black & white photos with just 100 lines of neural network code

왜 많이 안 읽어요?

Gesture Based Visually Writing System Using OpenCV and Python

Gesture Based Visually Writing System: Adding Visual User Interface

Gesture Based Visually Writing System: Adding Virtual Animationn, New Mode and New VUI

Gesture Based Visually Writing System: Add Slider, More Colors and Optimized OOP code

Gesture Based Visually Writing System: A Web App

Contour Based Game: Break The Bricks

Linear Regression from Scratch

Feed Forward Neural Network from Scratch

Convolutional Neural Networks from Scratch

Writing a Simple Image Processing Class from Scratch

Deploying a RASA Chatbot on Android using Unity3d

Naive Bayes for text classifications: Scratch to Framework

Simple OCR for Devanagari Handwritten Text

Reference

이 문제에 관하여(권적 신경 네트워크의 혼란: 그레이스케일에서 RGB로의 전환), 우리는 이곳에서 더 많은 자료를 발견하고 링크를 클릭하여 보았다 https://dev.to/qviper/messing-up-with-convolutional-neural-networks-grayscale-to-rgb-conversion-1opp

텍스트를 자유롭게 공유하거나 복사할 수 있습니다.하지만 이 문서의 URL은 참조 URL로 남겨 두십시오.

우수한 개발자 콘텐츠 발견에 전념 (Collection and Share based on the CC Protocol.)

NOI 2007 소셜네트워크서비스(COGS 15) Floyd 최단거리 및 시나리오 수

GhPython 코드 디버깅

좋은 웹페이지 즐겨찾기

개발자 우수 사이트 수집

개발자가 알아야 할 필수 사이트 100선 추천 우리는 당신을 위해 100개의 자주 사용하는 개발자 학습 사이트를 정리했습니다