CIFAR10 + 볼륨 신경 네트워크 TensorFlow
기사 디렉토리:
실행 플랫폼: Linux Python 버전: Python 3.6 TensorFlow 버전: 1.15.2 IDE: Colab
지난번의 전체 연결 네트워크를 토대로CIFAR10을 분류한 CIFAR10+전체 연결 네트워크TensorFlow(필요한 학생이 있으면 이것을 먼저 볼 수 있음) 일부 기본적인 부분, 예를 들어 스트레스 해소와 데이터 집합은 더 이상 소개하지 않겠습니다.
코드
데이터 세트 읽기
%tensorflow_version 1.x
import tensorflow as tf
import numpy as np
import time
import os
print(tf.__version__)
!/opt/bin/nvidia-smi
TensorFlow 1.x selected.
1.15.2
Mon May 11 13:18:03 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.67 Driver Version: 418.67 CUDA Version: 10.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla P100-PCIE... Off | 00000000:00:04.0 Off | 0 |
| N/A 52C P0 31W / 250W | 0MiB / 16280MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
#
def train_data_load(path):
import pickle
train_dataset = {'train_images':[], 'train_labels':[]}
for batch_num in range(1, 6):
with open('{0}/data_batch_{1}'.format(path, batch_num), 'rb') as f:
data = pickle.load(f, encoding='bytes')
train_dataset['train_images'].append(data[b'data'])
train_dataset['train_labels'].append(data[b'labels'])
train_dataset['train_images'] = np.concatenate(train_dataset['train_images'])
train_dataset['train_labels'] = np.concatenate(train_dataset['train_labels'])
return train_dataset
#
def test_data_load(path):
import pickle
test_dataset = {'test_images':[], 'test_labels':[]}
with open('{0}/test_batch'.format(path), 'rb') as f:
data = pickle.load(f, encoding='bytes')
test_dataset['test_images'] = data[b'data']
test_dataset['test_labels'] = data[b'labels']
return test_dataset
#
train_dataset = train_data_load('/content/drive/My Drive/cifar-10-batches-py')
test_dataset = test_data_load('/content/drive/My Drive/cifar-10-batches-py')
# reshape
train_images = np.reshape(train_dataset['train_images'], [-1,32,32,3])
train_labels = np.array(train_dataset['train_labels'])
test_images = np.reshape(test_dataset['test_images'], [-1,32,32,3])
test_labels = np.array(test_dataset['test_labels'])
#
print(train_images.shape)
print(train_labels.shape)
print(test_images.shape)
print(test_labels.shape)
(50000, 32, 32, 3)
(50000,)
(10000, 32, 32, 3)
(10000,)
권적 신경 네트워크 1
2층 권적층, 2층 최대 연못화층, 2층 전체 연결층#
images_placeholder = tf.placeholder(tf.float32, [None,32,32,3])
labels_placeholder = tf.placeholder(tf.int64, [None])
#
def weights_variable(shape, stddev):
return tf.Variable(tf.truncated_normal(shape, stddev=stddev))
#
def biases_variable(shape, stddev):
return tf.Variable(tf.truncated_normal(shape, stddev=stddev))
#
def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1,1,1,1], padding='SAME')
#
def max_pool(x):
return tf.nn.max_pool(x, ksize=[1,2,2,1], strides=[1,2,2,1], padding='SAME')
#
def LRnorm(x):
return tf.nn.lrn(x, 4, bias=1.0, alpha=0.001/9.0, beta=0.75)
# 1
weights_1 = weights_variable([3,3,3,10], 0.1)
biases_1 = biases_variable([10], 0.1)
conv_1 = conv2d(images_placeholder, weights_1)
relu_1 = tf.nn.relu(conv_1 + biases_1)
LRnorm_1 = LRnorm(relu_1)
print(LRnorm_1.shape)
# 2
weights_2 = weights_variable([3,3,10,10], 0.1)
biases_2 = biases_variable([10], 0.1)
conv_2 = conv2d(relu_1, weights_2)
relu_2 = tf.nn.relu(conv_2 + biases_2)
LRnorm_2 = LRnorm(relu_2)
max_pool_2 = max_pool(LRnorm_2)
print(max_pool_2.shape)
# 1
max_pool_2_flatten = tf.reshape(max_pool_2, [-1,16*16*10])
weights_fc1 = weights_variable([16*16*10,100], 0.1)
biases_fc1 = biases_variable([100], 0.1)
features_fc1 = tf.matmul(max_pool_2_flatten, weights_fc1) + biases_fc1
relu_fc1 = tf.nn.relu(features_fc1)
print(relu_fc1.shape)
# 2
weights_fc2 = weights_variable([100,10], 0.1)
biases_fc2 = biases_variable([10], 0.1)
features_fc2 = tf.matmul(relu_fc1, weights_fc2) + biases_fc2
print(features_fc2.shape)
(?, 32, 32, 10)
(?, 16, 16, 10)
(?, 100)
(?, 10)
#
batch_size = 2000
n_batch = 50000 // batch_size
#
loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=features_fc2, labels=labels_placeholder))
train_step = tf.train.GradientDescentOptimizer(3e-1).minimize(loss)
#
correct_prediction = tf.equal(tf.argmax(features_fc2, 1), labels_placeholder)
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
#
init = tf.global_variables_initializer()
#
with tf.Session() as sess:
sess.run(init)
for epoch in range(1, 1001):
if epoch % 100 == 0:
start_time = time.time()
for n in range(n_batch):
index = slice(n * batch_size, (n + 1) * batch_size)
images_batch, labels_batch = train_images[index], train_labels[index]
_, cross_loss = sess.run([train_step, loss], feed_dict={images_placeholder:images_batch, labels_placeholder:labels_batch})
acc = sess.run(accuracy, feed_dict={images_placeholder:test_images, labels_placeholder:test_labels})
if epoch % 100 == 0:
spend_time = time.time() - start_time
print('epoch {0}: accuracy={1}, loss={2}, spend_time={3}s'.format(epoch, acc, cross_loss, spend_time))
epoch 100: accuracy=0.29809999465942383, loss=1.9919211864471436, spend_time=1.129845142364502s
epoch 200: accuracy=0.32330000400543213, loss=1.9389268159866333, spend_time=1.1280453205108643s
epoch 300: accuracy=0.3481999933719635, loss=1.8605142831802368, spend_time=1.1415705680847168s
epoch 400: accuracy=0.3637000024318695, loss=1.9034373760223389, spend_time=1.1427974700927734s
epoch 500: accuracy=0.3601999878883362, loss=1.8122082948684692, spend_time=1.14522385597229s
epoch 600: accuracy=0.3691999912261963, loss=1.8035513162612915, spend_time=1.1477584838867188s
epoch 700: accuracy=0.3700999915599823, loss=1.804998517036438, spend_time=1.1441717147827148s
epoch 800: accuracy=0.3939000070095062, loss=1.7281118631362915, spend_time=1.13543701171875s
epoch 900: accuracy=0.3878999948501587, loss=1.754409909248352, spend_time=1.1523475646972656s
권적신경망2
위의 신경 네트워크의 정확도가 높지 않기 때문에 위의 네트워크를 수정하고dropout과 같이 의합을 줄인다.4개의 볼륨 층, 2개의 최대 연못 층, 2개의 전체 연결 층,dropout 사용#
images_placeholder = tf.placeholder(tf.float32, [None,32,32,3])
labels_placeholder = tf.placeholder(tf.int64, [None])
keep_prob = tf.placeholder(tf.float32)
#
def weights_variable(shape, stddev):
return tf.Variable(tf.truncated_normal(shape, stddev=stddev))
#
def biases_variable(shape, stddev):
return tf.Variable(tf.truncated_normal(shape, stddev=stddev))
#
def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1,1,1,1], padding='SAME')
#
def max_pool(x):
return tf.nn.max_pool(x, ksize=[1,2,2,1], strides=[1,2,2,1], padding='SAME')
# 1
weights_1_1 = weights_variable([3,3,3,32], 0.1)
biases_1_1 = biases_variable([32], 0.1)
conv_1_1 = conv2d(images_placeholder, weights_1_1) + biases_1_1
relu_1_1 = tf.nn.relu(conv_1_1)
dropout_1_1 = tf.nn.dropout(relu_1_1, keep_prob)
print(dropout_1_1.shape)
weights_1_2 = weights_variable([3,3,32,64], 0.1)
biases_1_2 = biases_variable([64], 0.1)
conv_1_2 = conv2d(relu_1_1, weights_1_2) + biases_1_2
dropout_1_2 = tf.nn.dropout(conv_1_2, keep_prob)
max_pool_1 = max_pool(dropout_1_2)
print(max_pool_1.shape)
# 2
weights_2_1 = weights_variable([3,3,64,32], 0.01)
biases_2_1 = biases_variable([32], 0.01)
conv_2_1 = conv2d(max_pool_1, weights_2_1) + biases_2_1
relu_2_1 = tf.nn.relu(conv_2_1)
dropout_2_1 = tf.nn.dropout(relu_2_1, keep_prob)
print(dropout_2_1.shape)
weights_2_2 = weights_variable([3,3,32,16], 0.01)
biases_2_2 = biases_variable([16], 0.01)
conv_2_2 = conv2d(relu_2_1, weights_2_2) + biases_2_2
relu_2_2 = tf.nn.relu(conv_2_2)
dropout_2_2 = tf.nn.dropout(relu_2_2, keep_prob)
max_pool_2 = max_pool(dropout_2_2)
print(max_pool_2.shape)
#
max_pool_flatten = tf.reshape(max_pool_2, [-1,8*8*16])
weights_3 = weights_variable([8*8*16,128], 0.01)
biases_3 = biases_variable([128], 0.01)
relu_3 = tf.nn.relu(tf.matmul(max_pool_flatten, weights_3) + biases_3)
print(relu_3.shape)
weights_4 = weights_variable([128,10], 0.01)
biases_4 = biases_variable([10], 0.01)
logits = tf.matmul(relu_3, weights_4) + biases_4
print(logits.shape)
(?, 32, 32, 32)
(?, 16, 16, 64)
(?, 16, 16, 32)
(?, 8, 8, 16)
(?, 128)
(?, 10)
#
batch_size = 2000
n_batch = 50000 // batch_size
#
loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits, labels=labels_placeholder))
train_step = tf.train.GradientDescentOptimizer(1.3e-3).minimize(loss)
#
correct_prediction = tf.equal(tf.argmax(logits, 1), labels_placeholder)
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
#
init = tf.global_variables_initializer()
#
with tf.Session() as sess:
sess.run(init)
total_start = time.time()
for epoch in range(1, 2501):
if epoch % 100 == 0:
start_time = time.time()
for n in range(n_batch):
index = slice(n * batch_size, (n + 1) * batch_size)
images_batch, labels_batch = train_images[index], train_labels[index]
_, cross_loss = sess.run([train_step, loss], feed_dict={images_placeholder:images_batch, labels_placeholder:labels_batch, keep_prob:0.5})
if epoch % 100 == 0:
acc = sess.run(accuracy, feed_dict={images_placeholder:test_images, labels_placeholder:test_labels, keep_prob:1.0})
train_acc = sess.run(accuracy, feed_dict={images_placeholder:train_images[:10000], labels_placeholder:train_labels[:10000], keep_prob:1.0})
spend_time = (time.time() - start_time)
print('epoch {0}: acc={1}, train_acc={2} loss={3}, spend_time={4}s'.format(
epoch, round(acc, 4), round(train_acc, 4), round(cross_loss, 4), round(spend_time, 4)))
print('total time:', time.time() - total_start)
!/opt/bin/nvidia-smi
epoch 100: acc=0.44440001249313354, train_acc=0.46230000257492065 loss=1.5571999549865723, spend_time=3.045s
epoch 200: acc=0.5058000087738037, train_acc=0.5239999890327454 loss=1.3623000383377075, spend_time=3.0011s
epoch 300: acc=0.5404000282287598, train_acc=0.5665000081062317 loss=1.2599999904632568, spend_time=2.9813s
epoch 400: acc=0.5759999752044678, train_acc=0.6057999730110168 loss=1.1812000274658203, spend_time=2.9894s
epoch 500: acc=0.5843999981880188, train_acc=0.6220999956130981 loss=1.087499976158142, spend_time=2.9768s
epoch 600: acc=0.5942000150680542, train_acc=0.6414999961853027 loss=1.0327999591827393, spend_time=2.966s
epoch 700: acc=0.6067000031471252, train_acc=0.6574000120162964 loss=0.9991999864578247, spend_time=2.9895s
epoch 800: acc=0.6279000043869019, train_acc=0.6883999705314636 loss=0.9380000233650208, spend_time=2.9679s
epoch 900: acc=0.628000020980835, train_acc=0.6955000162124634 loss=0.9157000184059143, spend_time=2.9899s
epoch 1000: acc=0.6384999752044678, train_acc=0.7186999917030334 loss=0.9246000051498413, spend_time=2.9645s
epoch 1100: acc=0.6434999704360962, train_acc=0.7261000275611877 loss=0.8478999733924866, spend_time=2.9789s
epoch 1200: acc=0.6438000202178955, train_acc=0.7386999726295471 loss=0.8327999711036682, spend_time=2.9629s
epoch 1300: acc=0.6460999846458435, train_acc=0.7479000091552734 loss=0.7997999787330627, spend_time=2.9818s
epoch 1400: acc=0.6478000283241272, train_acc=0.7581999897956848 loss=0.8011999726295471, spend_time=2.9604s
epoch 1500: acc=0.6498000025749207, train_acc=0.7641000151634216 loss=0.7519999742507935, spend_time=2.967s
epoch 1600: acc=0.6527000069618225, train_acc=0.7731000185012817 loss=0.7523000240325928, spend_time=2.9902s
epoch 1700: acc=0.652899980545044, train_acc=0.7773000001907349 loss=0.720300018787384, spend_time=2.9887s
epoch 1800: acc=0.6473000049591064, train_acc=0.7809000015258789 loss=0.6916999816894531, spend_time=2.9718s
epoch 1900: acc=0.6546000242233276, train_acc=0.7960000038146973 loss=0.692799985408783, spend_time=2.9756s
epoch 2000: acc=0.6424000263214111, train_acc=0.7843999862670898 loss=0.6625000238418579, spend_time=2.9677s
epoch 2100: acc=0.6549000144004822, train_acc=0.8105000257492065 loss=0.6502000093460083, spend_time=2.9705s
epoch 2200: acc=0.6466000080108643, train_acc=0.8051000237464905 loss=0.6184999942779541, spend_time=2.9707s
epoch 2300: acc=0.6388000249862671, train_acc=0.7962999939918518 loss=0.6225000023841858, spend_time=2.9724s
epoch 2400: acc=0.6539999842643738, train_acc=0.8223999738693237 loss=0.578499972820282, spend_time=2.9974s
epoch 2500: acc=0.6381000280380249, train_acc=0.8055999875068665 loss=0.6011999845504761, spend_time=3.0051s
total time: 6383.969125747681
Mon May 11 15:52:57 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.67 Driver Version: 418.67 CUDA Version: 10.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla P100-PCIE... Off | 00000000:00:04.0 Off | 0 |
| N/A 66C P0 43W / 250W | 9573MiB / 16280MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
+-----------------------------------------------------------------------------+
당면한 문제
%tensorflow_version 1.x
import tensorflow as tf
import numpy as np
import time
import os
print(tf.__version__)
!/opt/bin/nvidia-smi
TensorFlow 1.x selected.
1.15.2
Mon May 11 13:18:03 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.67 Driver Version: 418.67 CUDA Version: 10.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla P100-PCIE... Off | 00000000:00:04.0 Off | 0 |
| N/A 52C P0 31W / 250W | 0MiB / 16280MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
#
def train_data_load(path):
import pickle
train_dataset = {'train_images':[], 'train_labels':[]}
for batch_num in range(1, 6):
with open('{0}/data_batch_{1}'.format(path, batch_num), 'rb') as f:
data = pickle.load(f, encoding='bytes')
train_dataset['train_images'].append(data[b'data'])
train_dataset['train_labels'].append(data[b'labels'])
train_dataset['train_images'] = np.concatenate(train_dataset['train_images'])
train_dataset['train_labels'] = np.concatenate(train_dataset['train_labels'])
return train_dataset
#
def test_data_load(path):
import pickle
test_dataset = {'test_images':[], 'test_labels':[]}
with open('{0}/test_batch'.format(path), 'rb') as f:
data = pickle.load(f, encoding='bytes')
test_dataset['test_images'] = data[b'data']
test_dataset['test_labels'] = data[b'labels']
return test_dataset
#
train_dataset = train_data_load('/content/drive/My Drive/cifar-10-batches-py')
test_dataset = test_data_load('/content/drive/My Drive/cifar-10-batches-py')
# reshape
train_images = np.reshape(train_dataset['train_images'], [-1,32,32,3])
train_labels = np.array(train_dataset['train_labels'])
test_images = np.reshape(test_dataset['test_images'], [-1,32,32,3])
test_labels = np.array(test_dataset['test_labels'])
#
print(train_images.shape)
print(train_labels.shape)
print(test_images.shape)
print(test_labels.shape)
(50000, 32, 32, 3)
(50000,)
(10000, 32, 32, 3)
(10000,)
#
images_placeholder = tf.placeholder(tf.float32, [None,32,32,3])
labels_placeholder = tf.placeholder(tf.int64, [None])
#
def weights_variable(shape, stddev):
return tf.Variable(tf.truncated_normal(shape, stddev=stddev))
#
def biases_variable(shape, stddev):
return tf.Variable(tf.truncated_normal(shape, stddev=stddev))
#
def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1,1,1,1], padding='SAME')
#
def max_pool(x):
return tf.nn.max_pool(x, ksize=[1,2,2,1], strides=[1,2,2,1], padding='SAME')
#
def LRnorm(x):
return tf.nn.lrn(x, 4, bias=1.0, alpha=0.001/9.0, beta=0.75)
# 1
weights_1 = weights_variable([3,3,3,10], 0.1)
biases_1 = biases_variable([10], 0.1)
conv_1 = conv2d(images_placeholder, weights_1)
relu_1 = tf.nn.relu(conv_1 + biases_1)
LRnorm_1 = LRnorm(relu_1)
print(LRnorm_1.shape)
# 2
weights_2 = weights_variable([3,3,10,10], 0.1)
biases_2 = biases_variable([10], 0.1)
conv_2 = conv2d(relu_1, weights_2)
relu_2 = tf.nn.relu(conv_2 + biases_2)
LRnorm_2 = LRnorm(relu_2)
max_pool_2 = max_pool(LRnorm_2)
print(max_pool_2.shape)
# 1
max_pool_2_flatten = tf.reshape(max_pool_2, [-1,16*16*10])
weights_fc1 = weights_variable([16*16*10,100], 0.1)
biases_fc1 = biases_variable([100], 0.1)
features_fc1 = tf.matmul(max_pool_2_flatten, weights_fc1) + biases_fc1
relu_fc1 = tf.nn.relu(features_fc1)
print(relu_fc1.shape)
# 2
weights_fc2 = weights_variable([100,10], 0.1)
biases_fc2 = biases_variable([10], 0.1)
features_fc2 = tf.matmul(relu_fc1, weights_fc2) + biases_fc2
print(features_fc2.shape)
(?, 32, 32, 10)
(?, 16, 16, 10)
(?, 100)
(?, 10)
#
batch_size = 2000
n_batch = 50000 // batch_size
#
loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=features_fc2, labels=labels_placeholder))
train_step = tf.train.GradientDescentOptimizer(3e-1).minimize(loss)
#
correct_prediction = tf.equal(tf.argmax(features_fc2, 1), labels_placeholder)
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
#
init = tf.global_variables_initializer()
#
with tf.Session() as sess:
sess.run(init)
for epoch in range(1, 1001):
if epoch % 100 == 0:
start_time = time.time()
for n in range(n_batch):
index = slice(n * batch_size, (n + 1) * batch_size)
images_batch, labels_batch = train_images[index], train_labels[index]
_, cross_loss = sess.run([train_step, loss], feed_dict={images_placeholder:images_batch, labels_placeholder:labels_batch})
acc = sess.run(accuracy, feed_dict={images_placeholder:test_images, labels_placeholder:test_labels})
if epoch % 100 == 0:
spend_time = time.time() - start_time
print('epoch {0}: accuracy={1}, loss={2}, spend_time={3}s'.format(epoch, acc, cross_loss, spend_time))
epoch 100: accuracy=0.29809999465942383, loss=1.9919211864471436, spend_time=1.129845142364502s
epoch 200: accuracy=0.32330000400543213, loss=1.9389268159866333, spend_time=1.1280453205108643s
epoch 300: accuracy=0.3481999933719635, loss=1.8605142831802368, spend_time=1.1415705680847168s
epoch 400: accuracy=0.3637000024318695, loss=1.9034373760223389, spend_time=1.1427974700927734s
epoch 500: accuracy=0.3601999878883362, loss=1.8122082948684692, spend_time=1.14522385597229s
epoch 600: accuracy=0.3691999912261963, loss=1.8035513162612915, spend_time=1.1477584838867188s
epoch 700: accuracy=0.3700999915599823, loss=1.804998517036438, spend_time=1.1441717147827148s
epoch 800: accuracy=0.3939000070095062, loss=1.7281118631362915, spend_time=1.13543701171875s
epoch 900: accuracy=0.3878999948501587, loss=1.754409909248352, spend_time=1.1523475646972656s
#
images_placeholder = tf.placeholder(tf.float32, [None,32,32,3])
labels_placeholder = tf.placeholder(tf.int64, [None])
keep_prob = tf.placeholder(tf.float32)
#
def weights_variable(shape, stddev):
return tf.Variable(tf.truncated_normal(shape, stddev=stddev))
#
def biases_variable(shape, stddev):
return tf.Variable(tf.truncated_normal(shape, stddev=stddev))
#
def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1,1,1,1], padding='SAME')
#
def max_pool(x):
return tf.nn.max_pool(x, ksize=[1,2,2,1], strides=[1,2,2,1], padding='SAME')
# 1
weights_1_1 = weights_variable([3,3,3,32], 0.1)
biases_1_1 = biases_variable([32], 0.1)
conv_1_1 = conv2d(images_placeholder, weights_1_1) + biases_1_1
relu_1_1 = tf.nn.relu(conv_1_1)
dropout_1_1 = tf.nn.dropout(relu_1_1, keep_prob)
print(dropout_1_1.shape)
weights_1_2 = weights_variable([3,3,32,64], 0.1)
biases_1_2 = biases_variable([64], 0.1)
conv_1_2 = conv2d(relu_1_1, weights_1_2) + biases_1_2
dropout_1_2 = tf.nn.dropout(conv_1_2, keep_prob)
max_pool_1 = max_pool(dropout_1_2)
print(max_pool_1.shape)
# 2
weights_2_1 = weights_variable([3,3,64,32], 0.01)
biases_2_1 = biases_variable([32], 0.01)
conv_2_1 = conv2d(max_pool_1, weights_2_1) + biases_2_1
relu_2_1 = tf.nn.relu(conv_2_1)
dropout_2_1 = tf.nn.dropout(relu_2_1, keep_prob)
print(dropout_2_1.shape)
weights_2_2 = weights_variable([3,3,32,16], 0.01)
biases_2_2 = biases_variable([16], 0.01)
conv_2_2 = conv2d(relu_2_1, weights_2_2) + biases_2_2
relu_2_2 = tf.nn.relu(conv_2_2)
dropout_2_2 = tf.nn.dropout(relu_2_2, keep_prob)
max_pool_2 = max_pool(dropout_2_2)
print(max_pool_2.shape)
#
max_pool_flatten = tf.reshape(max_pool_2, [-1,8*8*16])
weights_3 = weights_variable([8*8*16,128], 0.01)
biases_3 = biases_variable([128], 0.01)
relu_3 = tf.nn.relu(tf.matmul(max_pool_flatten, weights_3) + biases_3)
print(relu_3.shape)
weights_4 = weights_variable([128,10], 0.01)
biases_4 = biases_variable([10], 0.01)
logits = tf.matmul(relu_3, weights_4) + biases_4
print(logits.shape)
(?, 32, 32, 32)
(?, 16, 16, 64)
(?, 16, 16, 32)
(?, 8, 8, 16)
(?, 128)
(?, 10)
#
batch_size = 2000
n_batch = 50000 // batch_size
#
loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=logits, labels=labels_placeholder))
train_step = tf.train.GradientDescentOptimizer(1.3e-3).minimize(loss)
#
correct_prediction = tf.equal(tf.argmax(logits, 1), labels_placeholder)
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
#
init = tf.global_variables_initializer()
#
with tf.Session() as sess:
sess.run(init)
total_start = time.time()
for epoch in range(1, 2501):
if epoch % 100 == 0:
start_time = time.time()
for n in range(n_batch):
index = slice(n * batch_size, (n + 1) * batch_size)
images_batch, labels_batch = train_images[index], train_labels[index]
_, cross_loss = sess.run([train_step, loss], feed_dict={images_placeholder:images_batch, labels_placeholder:labels_batch, keep_prob:0.5})
if epoch % 100 == 0:
acc = sess.run(accuracy, feed_dict={images_placeholder:test_images, labels_placeholder:test_labels, keep_prob:1.0})
train_acc = sess.run(accuracy, feed_dict={images_placeholder:train_images[:10000], labels_placeholder:train_labels[:10000], keep_prob:1.0})
spend_time = (time.time() - start_time)
print('epoch {0}: acc={1}, train_acc={2} loss={3}, spend_time={4}s'.format(
epoch, round(acc, 4), round(train_acc, 4), round(cross_loss, 4), round(spend_time, 4)))
print('total time:', time.time() - total_start)
!/opt/bin/nvidia-smi
epoch 100: acc=0.44440001249313354, train_acc=0.46230000257492065 loss=1.5571999549865723, spend_time=3.045s
epoch 200: acc=0.5058000087738037, train_acc=0.5239999890327454 loss=1.3623000383377075, spend_time=3.0011s
epoch 300: acc=0.5404000282287598, train_acc=0.5665000081062317 loss=1.2599999904632568, spend_time=2.9813s
epoch 400: acc=0.5759999752044678, train_acc=0.6057999730110168 loss=1.1812000274658203, spend_time=2.9894s
epoch 500: acc=0.5843999981880188, train_acc=0.6220999956130981 loss=1.087499976158142, spend_time=2.9768s
epoch 600: acc=0.5942000150680542, train_acc=0.6414999961853027 loss=1.0327999591827393, spend_time=2.966s
epoch 700: acc=0.6067000031471252, train_acc=0.6574000120162964 loss=0.9991999864578247, spend_time=2.9895s
epoch 800: acc=0.6279000043869019, train_acc=0.6883999705314636 loss=0.9380000233650208, spend_time=2.9679s
epoch 900: acc=0.628000020980835, train_acc=0.6955000162124634 loss=0.9157000184059143, spend_time=2.9899s
epoch 1000: acc=0.6384999752044678, train_acc=0.7186999917030334 loss=0.9246000051498413, spend_time=2.9645s
epoch 1100: acc=0.6434999704360962, train_acc=0.7261000275611877 loss=0.8478999733924866, spend_time=2.9789s
epoch 1200: acc=0.6438000202178955, train_acc=0.7386999726295471 loss=0.8327999711036682, spend_time=2.9629s
epoch 1300: acc=0.6460999846458435, train_acc=0.7479000091552734 loss=0.7997999787330627, spend_time=2.9818s
epoch 1400: acc=0.6478000283241272, train_acc=0.7581999897956848 loss=0.8011999726295471, spend_time=2.9604s
epoch 1500: acc=0.6498000025749207, train_acc=0.7641000151634216 loss=0.7519999742507935, spend_time=2.967s
epoch 1600: acc=0.6527000069618225, train_acc=0.7731000185012817 loss=0.7523000240325928, spend_time=2.9902s
epoch 1700: acc=0.652899980545044, train_acc=0.7773000001907349 loss=0.720300018787384, spend_time=2.9887s
epoch 1800: acc=0.6473000049591064, train_acc=0.7809000015258789 loss=0.6916999816894531, spend_time=2.9718s
epoch 1900: acc=0.6546000242233276, train_acc=0.7960000038146973 loss=0.692799985408783, spend_time=2.9756s
epoch 2000: acc=0.6424000263214111, train_acc=0.7843999862670898 loss=0.6625000238418579, spend_time=2.9677s
epoch 2100: acc=0.6549000144004822, train_acc=0.8105000257492065 loss=0.6502000093460083, spend_time=2.9705s
epoch 2200: acc=0.6466000080108643, train_acc=0.8051000237464905 loss=0.6184999942779541, spend_time=2.9707s
epoch 2300: acc=0.6388000249862671, train_acc=0.7962999939918518 loss=0.6225000023841858, spend_time=2.9724s
epoch 2400: acc=0.6539999842643738, train_acc=0.8223999738693237 loss=0.578499972820282, spend_time=2.9974s
epoch 2500: acc=0.6381000280380249, train_acc=0.8055999875068665 loss=0.6011999845504761, spend_time=3.0051s
total time: 6383.969125747681
Mon May 11 15:52:57 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 418.67 Driver Version: 418.67 CUDA Version: 10.1 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla P100-PCIE... Off | 00000000:00:04.0 Off | 0 |
| N/A 66C P0 43W / 250W | 9573MiB / 16280MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
+-----------------------------------------------------------------------------+
총결산
볼륨 신경 네트워크를 통해 CIFAR10을 분류한 결과 전체 연결 네트워크보다 비교적 큰 향상이 27%에서 65% 정도 높아졌다.하지만 2500번을 훈련했지만 인터넷은 1000번 정도에서 수렴되고 있다.그 다음에는 일부 온라인 모델을 사용하여 CIFAR10 분류의 정확성을 더욱 높일 것이다.
이 내용에 흥미가 있습니까?
현재 기사가 여러분의 문제를 해결하지 못하는 경우 AI 엔진은 머신러닝 분석(스마트 모델이 방금 만들어져 부정확한 경우가 있을 수 있음)을 통해 가장 유사한 기사를 추천합니다:
Faster R-CNN 코드의 anchors 분석
anchors는 proposal을 생성하는 rpn의 중점 내용 중 하나로서 Faster R-CNN에서 중점적으로 소개되었다.
다음은 anchors가 생성하는 부분 코드를 배워보자.나는 주로 그 중의 일부 중점 코드를...
텍스트를 자유롭게 공유하거나 복사할 수 있습니다.하지만 이 문서의 URL은 참조 URL로 남겨 두십시오.
CC BY-SA 2.5, CC BY-SA 3.0 및 CC BY-SA 4.0에 따라 라이센스가 부여됩니다.
이 내용에 흥미가 있습니까?
현재 기사가 여러분의 문제를 해결하지 못하는 경우 AI 엔진은 머신러닝 분석(스마트 모델이 방금 만들어져 부정확한 경우가 있을 수 있음)을 통해 가장 유사한 기사를 추천합니다:
Faster R-CNN 코드의 anchors 분석anchors는 proposal을 생성하는 rpn의 중점 내용 중 하나로서 Faster R-CNN에서 중점적으로 소개되었다. 다음은 anchors가 생성하는 부분 코드를 배워보자.나는 주로 그 중의 일부 중점 코드를...
텍스트를 자유롭게 공유하거나 복사할 수 있습니다.하지만 이 문서의 URL은 참조 URL로 남겨 두십시오.
CC BY-SA 2.5, CC BY-SA 3.0 및 CC BY-SA 4.0에 따라 라이센스가 부여됩니다.
좋은 웹페이지 즐겨찾기
