728x90
반응형
!git clone --depth 1 https://github.com/google/automl
!cd /content/automl/efficientdet; pip install -r requirements.txt
!nvidia-smi
Wed Dec 8 11:02:34 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 495.44 Driver Version: 460.32.03 CUDA Version: 11.2 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 Tesla K80 Off | 00000000:00:04.0 Off | 0 |
| N/A 72C P8 34W / 149W | 0MiB / 11441MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+
import os
import sys
import tensorflow.compat.v1 as tf
sys.path.append('/content/automl/efficientdet')
import hparams_config
from tf2 import anchors
from model_inspect import ModelInspectorPASCAL V0C 2007 데이터 세트 다운로드
# yolo 미러사이트에서 데이터셋 다운, pascal 공홈은 접근이 힘들어서
!wget http://pjreddie.com/media/files/VOCtrainval_06-Nov-2007.tar
!tar -xvf VOCtrainval_06-Nov-2007.tar > /dev/null 2>&1
--2021-12-08 11:03:44-- http://pjreddie.com/media/files/VOCtrainval_06-Nov-2007.tar
Resolving pjreddie.com (pjreddie.com)... 128.208.4.108
Connecting to pjreddie.com (pjreddie.com)|128.208.4.108|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://pjreddie.com/media/files/VOCtrainval_06-Nov-2007.tar [following]
--2021-12-08 11:03:45-- https://pjreddie.com/media/files/VOCtrainval_06-Nov-2007.tar
Connecting to pjreddie.com (pjreddie.com)|128.208.4.108|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 460032000 (439M) [application/octet-stream]
Saving to: ‘VOCtrainval_06-Nov-2007.tar’
VOCtrainval_06-Nov- 100%[===================>] 438.72M 21.1MB/s in 22s
2021-12-08 11:04:07 (20.4 MB/s) - ‘VOCtrainval_06-Nov-2007.tar’ saved [460032000/460032000]
!ls -lia /content/VOCdevkit/VOC2007/Annotations/*.xml| wc -l
# 5011학습 데이터와 검증 데이터를 tfrecord 형태로 변환
- google/automl/efficientdet/dataset/create_pascal_tfrecord.py
- create_pascal_tfrecord.py를 이용하여 XML 포맷의 Annotation을 tfrecord로 변환.
- create_pascal_tfrecord.py 는 ImageSet 디렉토리에 위치한 train.txt를 읽어서 해당 xml과 image를 train용 tfrecord로 변환. val.txt를 읽어서 valid용 tfrecord로 변환.
- train과 val용 각각 약 2500여개의 image/xml를 100개씩 하나의 tfrecord로 생성.
!mkdir -p /content/tfrecord/train
!mkdir -p /content/tfrecord/val
# train, val
# --output_path=/content/tfrecord/train/pascal에서 directory는 /content/tfrecord/train/ 까지, 뒤의 pascal을 tfrecord파일의 prefix임..
!cd /content/automl/efficientdet; PYTHONPATH="/content/automl/efficientdet:$PYTHONPATH" python dataset/create_pascal_tfrecord.py \
--data_dir=/content/VOCdevkit --year=VOC2007 --set=train --output_path=/content/tfrecord/train/pascal # pascal은 구분자
!cd /content/automl/efficientdet; PYTHONPATH="/content/automl/efficientdet:$PYTHONPATH" python dataset/create_pascal_tfrecord.py \
--data_dir=/content/VOCdevkit --year=VOC2007 --set=val --output_path=/content/tfrecord/val/pascal
# 버전 지정 # imageset의 main에서 찾음
# 한번에 100개씩 저장, image와 annotation을 함께 저장함
I1208 11:06:21.520007 139845113943936 create_pascal_tfrecord.py:254] Writing to output directory: /content/tfrecord/train
I1208 11:06:21.531928 139845113943936 create_pascal_tfrecord.py:287] Reading from PASCAL VOC2007 dataset.
I1208 11:06:21.532043 139845113943936 create_pascal_tfrecord.py:292] On image 0 of 2501
I1208 11:06:21.681272 139845113943936 create_pascal_tfrecord.py:292] On image 100 of 2501
I1208 11:06:21.820690 139845113943936 create_pascal_tfrecord.py:292] On image 200 of 2501
I1208 11:06:21.942792 139845113943936 create_pascal_tfrecord.py:292] On image 300 of 2501
I1208 11:06:22.065514 139845113943936 create_pascal_tfrecord.py:292] On image 400 of 2501
I1208 11:06:22.190493 139845113943936 create_pascal_tfrecord.py:292] On image 500 of 2501
I1208 11:06:22.317059 139845113943936 create_pascal_tfrecord.py:292] On image 600 of 2501
I1208 11:06:22.437084 139845113943936 create_pascal_tfrecord.py:292] On image 700 of 2501
I1208 11:06:22.564532 139845113943936 create_pascal_tfrecord.py:292] On image 800 of 2501
I1208 11:06:22.681631 139845113943936 create_pascal_tfrecord.py:292] On image 900 of 2501
I1208 11:06:22.803892 139845113943936 create_pascal_tfrecord.py:292] On image 1000 of 2501
I1208 11:06:22.930249 139845113943936 create_pascal_tfrecord.py:292] On image 1100 of 2501
I1208 11:06:23.092439 139845113943936 create_pascal_tfrecord.py:292] On image 1200 of 2501
I1208 11:06:23.220940 139845113943936 create_pascal_tfrecord.py:292] On image 1300 of 2501
I1208 11:06:23.388720 139845113943936 create_pascal_tfrecord.py:292] On image 1400 of 2501
I1208 11:06:23.519618 139845113943936 create_pascal_tfrecord.py:292] On image 1500 of 2501
I1208 11:06:23.643929 139845113943936 create_pascal_tfrecord.py:292] On image 1600 of 2501
I1208 11:06:23.772373 139845113943936 create_pascal_tfrecord.py:292] On image 1700 of 2501
I1208 11:06:23.902039 139845113943936 create_pascal_tfrecord.py:292] On image 1800 of 2501
I1208 11:06:24.024939 139845113943936 create_pascal_tfrecord.py:292] On image 1900 of 2501
I1208 11:06:24.143962 139845113943936 create_pascal_tfrecord.py:292] On image 2000 of 2501
I1208 11:06:24.280410 139845113943936 create_pascal_tfrecord.py:292] On image 2100 of 2501
I1208 11:06:24.428011 139845113943936 create_pascal_tfrecord.py:292] On image 2200 of 2501
I1208 11:06:24.549917 139845113943936 create_pascal_tfrecord.py:292] On image 2300 of 2501
I1208 11:06:24.677992 139845113943936 create_pascal_tfrecord.py:292] On image 2400 of 2501
I1208 11:06:24.799202 139845113943936 create_pascal_tfrecord.py:292] On image 2500 of 2501
I1208 11:06:29.488929 140697936627584 create_pascal_tfrecord.py:254] Writing to output directory: /content/tfrecord/val
I1208 11:06:29.501246 140697936627584 create_pascal_tfrecord.py:287] Reading from PASCAL VOC2007 dataset.
I1208 11:06:29.501367 140697936627584 create_pascal_tfrecord.py:292] On image 0 of 2510
I1208 11:06:29.630910 140697936627584 create_pascal_tfrecord.py:292] On image 100 of 2510
I1208 11:06:29.753052 140697936627584 create_pascal_tfrecord.py:292] On image 200 of 2510
I1208 11:06:29.876389 140697936627584 create_pascal_tfrecord.py:292] On image 300 of 2510
I1208 11:06:29.998543 140697936627584 create_pascal_tfrecord.py:292] On image 400 of 2510
I1208 11:06:30.122147 140697936627584 create_pascal_tfrecord.py:292] On image 500 of 2510
I1208 11:06:30.238847 140697936627584 create_pascal_tfrecord.py:292] On image 600 of 2510
I1208 11:06:30.364139 140697936627584 create_pascal_tfrecord.py:292] On image 700 of 2510
I1208 11:06:30.498464 140697936627584 create_pascal_tfrecord.py:292] On image 800 of 2510
I1208 11:06:30.622997 140697936627584 create_pascal_tfrecord.py:292] On image 900 of 2510
I1208 11:06:30.817349 140697936627584 create_pascal_tfrecord.py:292] On image 1000 of 2510
I1208 11:06:30.955173 140697936627584 create_pascal_tfrecord.py:292] On image 1100 of 2510
I1208 11:06:31.090512 140697936627584 create_pascal_tfrecord.py:292] On image 1200 of 2510
I1208 11:06:31.228125 140697936627584 create_pascal_tfrecord.py:292] On image 1300 of 2510
I1208 11:06:31.364045 140697936627584 create_pascal_tfrecord.py:292] On image 1400 of 2510
I1208 11:06:31.507602 140697936627584 create_pascal_tfrecord.py:292] On image 1500 of 2510
I1208 11:06:31.642575 140697936627584 create_pascal_tfrecord.py:292] On image 1600 of 2510
I1208 11:06:31.772114 140697936627584 create_pascal_tfrecord.py:292] On image 1700 of 2510
I1208 11:06:31.898878 140697936627584 create_pascal_tfrecord.py:292] On image 1800 of 2510
I1208 11:06:32.020886 140697936627584 create_pascal_tfrecord.py:292] On image 1900 of 2510
I1208 11:06:32.145466 140697936627584 create_pascal_tfrecord.py:292] On image 2000 of 2510
I1208 11:06:32.268044 140697936627584 create_pascal_tfrecord.py:292] On image 2100 of 2510
I1208 11:06:32.395010 140697936627584 create_pascal_tfrecord.py:292] On image 2200 of 2510
I1208 11:06:32.536167 140697936627584 create_pascal_tfrecord.py:292] On image 2300 of 2510
I1208 11:06:32.706319 140697936627584 create_pascal_tfrecord.py:292] On image 2400 of 2510
I1208 11:06:32.842475 140697936627584 create_pascal_tfrecord.py:292] On image 2500 of 2510Train용 config 설정.
- 학습을 위한 다양한 설정을 config로 저장. model은 efficientdet-d0 로 적용.
# epochs시마다 학습된 weight파일을 저장한 디렉토리 Google drive로 설정.
# Google Drive 접근을 위한 Mount 적용.
import os, sys
from google.colab import drive
drive.mount('/content/gdrive')
# soft link로 Google Drive Directory 연결.
!ln -s /content/gdrive/My\ Drive/ /mydrive
!ls /mydrive
!mkdir -p /mydrive/model_trained
Mounted at /content/gdrive
config = hparams_config.get_detection_config('efficientdet-d0')
print(config)
# mmdetection은 이미 default 값이 있고 그걸 업데이터 하는 방식
# automl은 default.config를 직접 채우는 방식
act_type: swish
alpha: 0.25
anchor_scale: 4.0
apply_bn_for_resampling: true
aspect_ratios:
- 1.0
- 2.0
- 0.5
autoaugment_policy: null
backbone_config: null
backbone_name: efficientnet-b0
box_class_repeats: 3
box_loss_weight: 50.0
ckpt_var_scope: null
clip_gradients_norm: 10.0
conv_after_downsample: false
conv_bn_act_pattern: false
data_format: channels_last
dataset_type: null
delta: 0.1
drop_remainder: true
first_lr_drop_epoch: 200.0
fpn_cell_repeats: 3
fpn_config: null
fpn_name: null
fpn_num_filters: 64
fpn_weight_method: null
gamma: 1.5
grad_checkpoint: false
grid_mask: false
heads:
- object_detection
image_size: 512
img_summary_steps: null
input_rand_hflip: true
iou_loss_type: null
iou_loss_weight: 1.0
is_training_bn: true
jitter_max: 2.0
jitter_min: 0.1
label_map: null
label_smoothing: 0.0
learning_rate: 0.08
loss_scale: null
lr_decay_method: cosine
lr_warmup_epoch: 1.0
lr_warmup_init: 0.008
map_freq: 5
max_instances_per_image: 100
max_level: 7
mean_rgb:
- 123.675
- 116.28
- 103.53
min_level: 3
mixed_precision: false
model_optimizations: {}
momentum: 0.9
moving_average_decay: 0.9998
name: efficientdet-d0
nms_configs:
iou_thresh: null
max_nms_inputs: 0
max_output_size: 100
method: gaussian
pyfunc: false
score_thresh: 0.0
sigma: null
num_classes: 90
num_epochs: 300
num_scales: 3
optimizer: sgd
poly_lr_power: 0.9
positives_momentum: null
regenerate_source_id: false
sample_image: null
save_freq: epoch
scale_range: false
second_lr_drop_epoch: 250.0
seg_num_classes: 3
separable_conv: true
skip_crowd_during_training: true
skip_mismatch: true
stddev_rgb:
- 58.395
- 57.120000000000005
- 57.375
strategy: null
survival_prob: null
target_size: null
tflite_max_detections: 100
use_keras_model: true
var_freeze_expr: null
verbose: 1
weight_decay: 4.0e-05
class TRAIN_CFG:
model_name = 'efficientdet-d0' # efficientdet 모델명
strategy = '' # tpu, 여러개의 GPU들, 단일 GPU 일때 학습 strategy 설정.
model_dir = '/mydrive/model_trained' # 학습된 모델이 저장될 위치, 학습할때 callbacks에서 모델체크포인트가 돌면서 저장함
pretrained_ckpt = '/content/efficientdet-d0' # download 할 pretrained 모델
hparams = 'num_classes=20,moving_average_decay=0,mixed_precision=true' # 한번에 쓰려고
use_xla = False
use_fake_data = False
batch_size = 8
eval_samples = 5000 # evaluation image 데이터 갯수
steps_per_execution = 1 # ModelCheckPoint의 save_freq 를 숫자로 설정할 경우 사용.
num_examples_per_epoch = 2500 # 1 epochs 시 적용하는 examples 개수 ( 한record를 example이라 표현함
num_epochs = 15 # epochs 횟수
train_file_pattern = '/content/tfrecord/train/pascal-*.tfrecord' # 학습용 tfrecords를 glob 형태로 가져오는 표현식.
val_file_pattern = '/content/tfrecord/val/pascal-*.tfrecord' # 검증용 tfrecords를 glob 형태로 가져오는 표현식.
val_json_file = None # optional coco validation json,
mode = 'traineval' # train만 적용 또는 train과 eval함께 적용(traineval)
# 나중에 config에 다 들어가지만 flags에 있는 인자를 옮겨온 것
num_cores = 2 # tpu 8 일때 적용.
tpu = None
gcp_project = None
tpu_zone = None
eval_master = ''
eval_name = None
tf_random_seed = 2021
profile = False
debug = False
from tf2.train import setup_model
import hparams_config
import utils
from tf2 import tfmot
from tf2 import train_lib
from tf2 import util_keras
config = hparams_config.get_detection_config(TRAIN_CFG.model_name)
config.override(TRAIN_CFG.hparams)
steps_per_epoch = TRAIN_CFG.num_examples_per_epoch // TRAIN_CFG.batch_size
if tf.config.list_physical_devices('GPU'):
ds_strategy = tf.distribute.OneDeviceStrategy('device:GPU:0')
else:
ds_strategy = tf.distribute.OneDeviceStrategy('device:CPU:0')
print(ds_strategy)
#steps_per_execution은 ModelCheckpoint의 save_freq를 숫자로 설정할 시 적용. num_epochs, steps_per_epoch는 추후에 model.fit()에서 설정되지만, 여기서는 일단 값을 설정해야함.
params = dict(
profile=TRAIN_CFG.profile,
mode = TRAIN_CFG.mode,
model_name=TRAIN_CFG.model_name,
steps_per_execution=TRAIN_CFG.steps_per_execution,
num_epochs = TRAIN_CFG.num_epochs,
model_dir=TRAIN_CFG.model_dir,
steps_per_epoch=steps_per_epoch,
strategy=TRAIN_CFG.strategy,
batch_size=TRAIN_CFG.batch_size,
tf_random_seed=TRAIN_CFG.tf_random_seed,
debug=TRAIN_CFG.debug,
val_json_file=TRAIN_CFG.val_json_file,
eval_samples=TRAIN_CFG.eval_samples,
num_shards=ds_strategy.num_replicas_in_sync
)
config.override(params, True)
# image size를 tuple 형태로 변환. 512는 (512, 512)로 '1920x880' 은 (1920, 880) 으로 변환.
config.image_size = utils.parse_image_size(config.image_size)
print(config)
<tensorflow.python.distribute.one_device_strategy.OneDeviceStrategyV1 object at 0x7f12d2237990>
act_type: swish
alpha: 0.25
anchor_scale: 4.0
apply_bn_for_resampling: true
aspect_ratios:
- 1.0
- 2.0
- 0.5
autoaugment_policy: null
backbone_config: null
backbone_name: efficientnet-b0
batch_size: 8
box_class_repeats: 3
box_loss_weight: 50.0
ckpt_var_scope: null
clip_gradients_norm: 10.0
conv_after_downsample: false
conv_bn_act_pattern: false
data_format: channels_last
dataset_type: null
debug: false
delta: 0.1
drop_remainder: true
eval_samples: 5000
first_lr_drop_epoch: 200.0
fpn_cell_repeats: 3
fpn_config: null
fpn_name: null
fpn_num_filters: 64
fpn_weight_method: null
gamma: 1.5
grad_checkpoint: false
grid_mask: false
heads:
- object_detection
image_size: !!python/tuple
- 512
- 512
img_summary_steps: null
input_rand_hflip: true
iou_loss_type: null
iou_loss_weight: 1.0
is_training_bn: true
jitter_max: 2.0
jitter_min: 0.1
label_map: null
label_smoothing: 0.0
learning_rate: 0.08
loss_scale: null
lr_decay_method: cosine
lr_warmup_epoch: 1.0
lr_warmup_init: 0.008
map_freq: 5
max_instances_per_image: 100
max_level: 7
mean_rgb:
- 123.675
- 116.28
- 103.53
min_level: 3
mixed_precision: true
mode: traineval
model_dir: /mydrive/model_trained
model_name: efficientdet-d0
model_optimizations: {}
momentum: 0.9
moving_average_decay: 0
name: efficientdet-d0
nms_configs:
iou_thresh: null
max_nms_inputs: 0
max_output_size: 100
method: gaussian
pyfunc: false
score_thresh: 0.0
sigma: null
num_classes: 20
num_epochs: 15
num_scales: 3
num_shards: 1
optimizer: sgd
poly_lr_power: 0.9
positives_momentum: null
profile: false
regenerate_source_id: false
sample_image: null
save_freq: epoch
scale_range: false
second_lr_drop_epoch: 250.0
seg_num_classes: 3
separable_conv: true
skip_crowd_during_training: true
skip_mismatch: true
stddev_rgb:
- 58.395
- 57.120000000000005
- 57.375
steps_per_epoch: 312
steps_per_execution: 1
strategy: ''
survival_prob: null
target_size: null
tf_random_seed: 2021
tflite_max_detections: 100
use_keras_model: true
val_json_file: null
var_freeze_expr: null
verbose: 1
weight_decay: 4.0e-05
Model 생성
- Config를 기반으로 EfficientDet d0 모델을 생성
- Coco Pretrained 파일을 다운로드 한 뒤 이 checkpoint파일의 weight를 생성한 d0 모델로 로딩
import utils
from tf2 import tfmot
from tf2 import train_lib
from tf2 import util_keras
# P100 GPU Card에서는 아래 수행하지 말것. V100 GPU 시에는 mixed_float16으로 mixed_precision 설정.
#precision = utils.get_precision(config.strategy, config.mixed_precision)
#policy = tf.keras.mixed_precision.Policy(precision)
#tf.keras.mixed_precision.set_global_policy(policy)
MODEL = 'efficientdet-d0'
def download(m):
if m not in os.listdir():
!wget https://storage.googleapis.com/cloud-tpu-checkpoints/efficientdet/coco/{m}.tar.gz
!tar zxf {m}.tar.gz
ckpt_path = os.path.join(os.getcwd(), m)
return ckpt_path
# Download checkpoint.
ckpt_path = download(MODEL)
print('Use model in {}'.format(ckpt_path))
--2021-12-08 11:08:54-- https://storage.googleapis.com/cloud-tpu-checkpoints/efficientdet/coco/efficientdet-d0.tar.gz
Resolving storage.googleapis.com (storage.googleapis.com)... 74.125.133.128, 74.125.140.128, 108.177.15.128, ...
Connecting to storage.googleapis.com (storage.googleapis.com)|74.125.133.128|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 28994253 (28M) [application/octet-stream]
Saving to: ‘efficientdet-d0.tar.gz’
efficientdet-d0.tar 100%[===================>] 27.65M 177MB/s in 0.2s
2021-12-08 11:08:54 (177 MB/s) - ‘efficientdet-d0.tar.gz’ saved [28994253/28994253]
Use model in /content/efficientdet-d0coco pretrained model의 num=80을 num_class=20으로 바꿔야 함, 심지어 다운된 모델을 90개를 가지고 있음
from tf2 import train_lib
from tf2 import train
# 20개의 class를 가진 efficientdet d0 모델을 생성.
model = train_lib.EfficientDetNetTrain(config=config)
model = train.setup_model(model, config)
# 만약 pretrained 모델이 있으면, 해당 checkpoint weight를 모델로 로딩. 이때 classification layer는 제외.
# transfer learning을 위해 classification layer는 80개는 가져오면 안됨
# last checkpoint를 읽어서 restore check point를 하되, exclude_layers는 class_net이다
#class TRAIN_CFG: pretrained_ckpt = '/content/efficientdet-d0'
if TRAIN_CFG.pretrained_ckpt:
ckpt_path = tf.train.latest_checkpoint(TRAIN_CFG.pretrained_ckpt)
util_keras.restore_ckpt(
model,
ckpt_path,
config.moving_average_decay,
exclude_layers=['class_net'])
train.init_experimental(config)
model.summary()
/content/automl/efficientdet/utils.py:23: UserWarning: `layer.updates` will be removed in a future version. This property should not be used in TensorFlow 2.0, as `updates` are applied automatically.
from tensorflow.python.tpu import tpu_function # pylint:disable=g-direct-tensorflow-import
/content/automl/efficientdet/utils.py:255: UserWarning: `layer.updates` will be removed in a future version. This property should not be used in TensorFlow 2.0, as `updates` are applied automatically.
for u in self.updates:
WARNING:absl:Shape mismatch: class_net/class-predict/pointwise_kernel
WARNING:absl:Shape mismatch: class_net/class-predict/bias
Model: ""
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
efficientnet-b0 (Model) multiple 3634844
resample_p6 (ResampleFeatur multiple 20800
eMap)
resample_p7 (ResampleFeatur multiple 0
eMap)
fpn_cells (FPNCells) multiple 179321
class_net (ClassNet) multiple 30324
box_net (BoxNet) multiple 20964
=================================================================
Total params: 3,886,253
Trainable params: 3,839,117
Non-trainable params: 47,136
_________________________________________________________________학습과 검증용 Dataset을 생성하고, Train 수행.
- 학습과 검증 데이터용 dataset 생성을 위한 get_dataset() 함수 생성.
'''
Class TRAIN_CFG:
train_file_pattern = '/content/tfrecord/train/pascal-*.tfrecord' # 학습용 tfrecords를 glob 형태로 가져오는 표현식.
val_file_pattern = '/content/tfrecord/val/pascal-*.tfrecord' # 검증용 tfrecords를 glob 형태로 가져오는 표현식.
'''
import dataloader
def get_dataset(is_training, config):
# is_training이 True이면 TRAIN_CFG의 train_file_pattern, 그렇지 아니면 val_file_pattern
file_pattern = (
TRAIN_CFG.train_file_pattern
if is_training else TRAIN_CFG.val_file_pattern) # 만들어진 tfrecord에서 train, val을 지정
if not file_pattern:
raise ValueError('No matching files.')
return dataloader.InputReader( # 읽혀진 데이터셋의 패턴을 모델에 집어넣어주는 역할
file_pattern,
is_training=is_training,
use_fake_data=TRAIN_CFG.use_fake_data,
max_instances_per_image=config.max_instances_per_image,
debug=TRAIN_CFG.debug)(
config.as_dict())
import pandas as pd
# train.txt와 val.txt를 읽어서 train과 val 용 image 건수를 구함
train_df = pd.read_csv('/content/VOCdevkit/VOC2007/ImageSets/Main/train.txt', sep=' ',
header=None, names=['file_id'], dtype={'file_id':str})
val_df = pd.read_csv('/content/VOCdevkit/VOC2007/ImageSets/Main/val.txt', sep=' ',
header=None, names=['file_id'], dtype={'file_id':str})
train_images_num = train_df.shape[0]
val_images_num = val_df.shape[0]
print(train_images_num, val_images_num)
train_df.head()
2501 2510
file_id
0 000012
1 000017
2 000023
3 000026
4 000032
import tensorflow as tf
from tf2 import train_lib
from tf2 import train
# config에 기반하여 모델을 생성하고 pretrained weight를 로딩하는 함수 생성.
# 바꿀 때 config 수정하면 됨
def get_efficientdet_model(config):
model = train_lib.EfficientDetNetTrain(config=config)
model = train.setup_model(model, config)
if TRAIN_CFG.pretrained_ckpt:
ckpt_path = tf.train.latest_checkpoint(TRAIN_CFG.pretrained_ckpt)
util_keras.restore_ckpt(
model,
ckpt_path,
config.moving_average_decay,
exclude_layers=['class_net'])
train.init_experimental(config)
return model
model = get_efficientdet_model(config)
model.summary()
/content/automl/efficientdet/utils.py:23: UserWarning: `layer.updates` will be removed in a future version. This property should not be used in TensorFlow 2.0, as `updates` are applied automatically.
from tensorflow.python.tpu import tpu_function # pylint:disable=g-direct-tensorflow-import
/content/automl/efficientdet/utils.py:255: UserWarning: `layer.updates` will be removed in a future version. This property should not be used in TensorFlow 2.0, as `updates` are applied automatically.
for u in self.updates:
WARNING:absl:Not found efficientnet-b0/stem_1/conv2d_1/kernel in /content/efficientdet-d0/model
WARNING:absl:Not found efficientnet-b0/stem_1/tpu_batch_normalization/gamma in /content/efficientdet-d0/model
WARNING:absl:Not found efficientnet-b0/stem_1/tpu_batch_normalization/beta in /content/efficientdet-d0/model
WARNING:absl:Shape mismatch: class_net/class-predict/pointwise_kernel
WARNING:absl:Shape mismatch: class_net/class-predict/bias
WARNING:absl:Not found efficientnet-b0/stem_1/tpu_batch_normalization/moving_mean in /content/efficientdet-d0/model
WARNING:absl:Not found efficientnet-b0/stem_1/tpu_batch_normalization/moving_variance in /content/efficientdet-d0/model
Model: ""
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
efficientnet-b0 (Model) multiple 3634844
resample_p6 (ResampleFeatur multiple 20800
eMap)
resample_p7 (ResampleFeatur multiple 0
eMap)
fpn_cells (FPNCells) multiple 179321
class_net (ClassNet) multiple 30324
box_net (BoxNet) multiple 20964
=================================================================
Total params: 3,886,253
Trainable params: 3,839,117
Non-trainable params: 47,136
_________________________________________________________________
from tf2 import train
import numpy as np
# config에 설정된 steps_per_epoch, num_epochs는 무시하고 여기서 새로 설정.
# steps_per_epoch는 전체 학습데이터 이미지 건수//batch_size, val_steps_per_epoch는 전체 검증 데이터 이미지 건수//batch_size
tr_steps_per_epoch = train_images_num//config.batch_size
val_steps_per_epoch = val_images_num//config.batch_size
print('tr_steps_per_epoch:', tr_steps_per_epoch, 'val_steps_per_epoch:', val_steps_per_epoch)
# config.mode가 traineval 또는 eval일 경우 검증 dataset 생성.
val_dataset = get_dataset(False, config) if 'eval' in config.mode else None
#callback은 config에 설정된 구성대로 생성. ModelCheckpoint는 epoch시마다, COCO Evaluation는 5회 epoch시마다 수행됨.
#config.save_freq = eval;config.map_freq = 5
# 1 epoch시마다 P100에서 약 3분30초 걸림. 적절한 epochs 수 설정 필요.
model.fit(
get_dataset(True, config), # train dataset
epochs=15,
steps_per_epoch=tr_steps_per_epoch ,
callbacks=train_lib.get_callbacks(config.as_dict(), val_dataset),
validation_data=val_dataset,
validation_steps=val_steps_per_epoch)
tf.keras.backend.clear_session()
tr_steps_per_epoch: 312 val_steps_per_epoch: 313
Epoch 1/15
/content/automl/efficientdet/utils.py:23: UserWarning: `layer.updates` will be removed in a future version. This property should not be used in TensorFlow 2.0, as `updates` are applied automatically.
from tensorflow.python.tpu import tpu_function # pylint:disable=g-direct-tensorflow-import
/content/automl/efficientdet/utils.py:255: UserWarning: `layer.updates` will be removed in a future version. This property should not be used in TensorFlow 2.0, as `updates` are applied automatically.
for u in self.updates:
312/312 [==============================] - ETA: 0s - det_loss: 129.3025 - cls_loss: 129.0228 - box_loss: 0.0056 - reg_l2_loss: 0.0954 - loss: 129.3978 - learning_rate: 0.0090 - gradient_norm: 3.3109
Epoch 00001: saving model to /mydrive/model_trained/ckpt-1
312/312 [==============================] - 450s 1s/step - det_loss: 128.8929 - cls_loss: 128.6131 - box_loss: 0.0056 - reg_l2_loss: 0.0954 - loss: 128.9883 - learning_rate: 0.0090 - gradient_norm: 3.3069 - val_det_loss: 0.9879 - val_cls_loss: 0.6648 - val_box_loss: 0.0065 - val_reg_l2_loss: 0.0955 - val_loss: 1.0834
Epoch 2/15
312/312 [==============================] - ETA: 0s - det_loss: 0.8750 - cls_loss: 0.6585 - box_loss: 0.0043 - reg_l2_loss: 0.0956 - loss: 0.9706 - learning_rate: 0.0097 - gradient_norm: 2.6721
Epoch 00002: saving model to /mydrive/model_trained/ckpt-2
312/312 [==============================] - 387s 1s/step - det_loss: 0.8749 - cls_loss: 0.6584 - box_loss: 0.0043 - reg_l2_loss: 0.0956 - loss: 0.9705 - learning_rate: 0.0097 - gradient_norm: 2.6714 - val_det_loss: 0.8318 - val_cls_loss: 0.5513 - val_box_loss: 0.0056 - val_reg_l2_loss: 0.0957 - val_loss: 0.9275
Epoch 3/15
312/312 [==============================] - ETA: 0s - det_loss: 0.7677 - cls_loss: 0.5700 - box_loss: 0.0040 - reg_l2_loss: 0.0958 - loss: 0.8634 - learning_rate: 0.0092 - gradient_norm: 2.7775
Epoch 00003: saving model to /mydrive/model_trained/ckpt-3
312/312 [==============================] - 401s 1s/step - det_loss: 0.7678 - cls_loss: 0.5701 - box_loss: 0.0040 - reg_l2_loss: 0.0958 - loss: 0.8636 - learning_rate: 0.0092 - gradient_norm: 2.7790 - val_det_loss: 0.7571 - val_cls_loss: 0.4841 - val_box_loss: 0.0055 - val_reg_l2_loss: 0.0959 - val_loss: 0.8529학습된 모델 파일을 이용하여 Inference 수행.
import hparams_config
infer_config = hparams_config.get_efficientdet_config('efficientdet-d0')
print(infer_config)
act_type: swish
alpha: 0.25
anchor_scale: 4.0
apply_bn_for_resampling: true
aspect_ratios:
- 1.0
- 2.0
- 0.5
autoaugment_policy: null
backbone_config: null
backbone_name: efficientnet-b0
box_class_repeats: 3
box_loss_weight: 50.0
ckpt_var_scope: null
clip_gradients_norm: 10.0
conv_after_downsample: false
conv_bn_act_pattern: false
data_format: channels_last
dataset_type: null
delta: 0.1
drop_remainder: true
first_lr_drop_epoch: 200.0
fpn_cell_repeats: 3
fpn_config: null
fpn_name: null
fpn_num_filters: 64
fpn_weight_method: null
gamma: 1.5
grad_checkpoint: false
grid_mask: false
heads:
- object_detection
image_size: 512
img_summary_steps: null
input_rand_hflip: true
iou_loss_type: null
iou_loss_weight: 1.0
is_training_bn: true
jitter_max: 2.0
jitter_min: 0.1
label_map: null
label_smoothing: 0.0
learning_rate: 0.08
loss_scale: null
lr_decay_method: cosine
lr_warmup_epoch: 1.0
lr_warmup_init: 0.008
map_freq: 5
max_instances_per_image: 100
max_level: 7
mean_rgb:
- 123.675
- 116.28
- 103.53
min_level: 3
mixed_precision: false
model_optimizations: {}
momentum: 0.9
moving_average_decay: 0.9998
name: efficientdet-d0
nms_configs:
iou_thresh: null
max_nms_inputs: 0
max_output_size: 100
method: gaussian
pyfunc: false
score_thresh: 0.0
sigma: null
num_classes: 90
num_epochs: 300
num_scales: 3
optimizer: sgd
poly_lr_power: 0.9
positives_momentum: null
regenerate_source_id: false
sample_image: null
save_freq: epoch
scale_range: false
second_lr_drop_epoch: 250.0
seg_num_classes: 3
separable_conv: true
skip_crowd_during_training: true
skip_mismatch: true
stddev_rgb:
- 58.395
- 57.120000000000005
- 57.375
strategy: null
survival_prob: null
target_size: null
tflite_max_detections: 100
use_keras_model: true
var_freeze_expr: null
verbose: 1
weight_decay: 4.0e-05
infer_config = hparams_config.get_efficientdet_config('efficientdet-d0')
# config의 특정 항목을 update
infer_config.model_name = 'efficientdet-d0'
infer_config.model_dir = '/mydrive/model_trained'
# infer_config의 num_classes는 20로 바뀌어야 함.
infer_config.num_classes =20 # 여길 바꿔줘야함
infer_config.is_training_bn = False
infer_config.nms_configs.score_thresh = 0.4
infer_config.nms_configs.max_output_size = 100
import inference
from keras import efficientdet_keras
model = efficientdet_keras.EfficientDetModel(config=infer_config)
model.build((None, None, None, 3))
print('#### checkpoint name:', tf.train.latest_checkpoint(infer_config.model_dir))
model.load_weights(tf.train.latest_checkpoint(infer_config.model_dir))
model.summary()
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/parallel_for/pfor.py:2382: calling gather (from tensorflow.python.ops.array_ops) with validate_indices is deprecated and will be removed in a future version.
Instructions for updating:
The `validate_indices` argument has no effect. Indices are always validated on CPU and never validated on GPU.
WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/parallel_for/pfor.py:2382: calling gather (from tensorflow.python.ops.array_ops) with validate_indices is deprecated and will be removed in a future version.
Instructions for updating:
The `validate_indices` argument has no effect. Indices are always validated on CPU and never validated on GPU.
WARNING:tensorflow:Using a while_loop for converting ResizeBilinear
WARNING:tensorflow:Using a while_loop for converting ResizeBilinear
/usr/local/lib/python3.7/dist-packages/tensorflow/python/keras/engine/base_layer.py:1331: UserWarning: `layer.updates` will be removed in a future version. This property should not be used in TensorFlow 2.0, as `updates` are applied automatically.
warnings.warn('`layer.updates` will be removed in a future version. '
WARNING:tensorflow:Using a while_loop for converting NonMaxSuppressionV5
WARNING:tensorflow:Using a while_loop for converting NonMaxSuppressionV5
#### checkpoint name: /mydrive/model_trained/ckpt-15
Model: ""
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
efficientnet-b0 (Model) multiple 3634844
_________________________________________________________________
resample_p6 (ResampleFeature multiple 20800
_________________________________________________________________
resample_p7 (ResampleFeature multiple 0
_________________________________________________________________
fpn_cells (FPNCells) multiple 179321
_________________________________________________________________
class_net (ClassNet) multiple 30324
_________________________________________________________________
box_net (BoxNet) multiple 20964
=================================================================
Total params: 3,886,253
Trainable params: 3,839,117
Non-trainable params: 47,136
_________________________________________________________________
import time
class ExportModel(tf.Module):
def __init__(self, model):
super().__init__()
self.model = model
@tf.function
def f(self, imgs):
#model(imgs, training=False, post_mode='global')
return self.model(imgs, training=False, post_mode='global')
export_model = ExportModel(model)
!mkdir -p /content/data
!wget -O ./data/beatles01.jpg https://raw.githubusercontent.com/chulminkw/DLCV/master/data/image/beatles01.jpg
!wget -O ./data/baseball01.jpg https://raw.githubusercontent.com/chulminkw/DLCV/master/data/image/baseball01.jpg
import cv2
import matplotlib.pyplot as plt
import numpy as np
img = cv2.cvtColor(cv2.imread('/content/data/beatles01.jpg'), cv2.COLOR_BGR2RGB)
imgs= img[np.newaxis, ...]
start_time = time.time()
boxes, scores, classes, valid_len = export_model.f(imgs)
print('elapsed time:', time.time() - start_time)
# elapsed time: 0.021811485290527344
labels_to_names = {1:'aeroplane', 2:'bicycle', 3:'bird', 4:'boat', 5:'bottle', 6:'bus', 7:'car',
8:'cat', 9:'chair', 10:'cow', 11:'diningtable', 12:'dog', 13:'horse',
14:'motorbike', 15:'person', 16:'pottedplant', 17:'sheep', 18:'sofa', 19:'train',
20:'tvmonitor'}
def get_detected_img(export_model, img_array, is_print=True):
# automl efficent은 반환 bbox 좌표값이 원본 이미지 좌표값으로 되어 있으므로 별도의 scaling작업 필요 없음.
'''
height = img_array.shape[0]
width = img_array.shape[1]
'''
# cv2의 rectangle()은 인자로 들어온 이미지 배열에 직접 사각형을 업데이트 하므로 그림 표현을 위한 별도의 이미지 배열 생성.
draw_img = img_array.copy()
# bounding box의 테두리와 caption 글자색 지정
green_color=(0, 255, 0)
red_color=(0, 0, 255)
# cv2로 만들어진 numpy image array를 tensor로 변환
img_tensor = tf.convert_to_tensor(img_array, dtype=tf.uint8)[tf.newaxis, ...]
#img_tensor = tf.convert_to_tensor(img_array, dtype=tf.float32)[tf.newaxis, ...]
# efficientdet 모델을 다운로드 한 뒤 inference 수행.
start_time = time.time()
# automl efficientdet 모델은 boxes, score, classes, num_detections를 각각 Tensor로 반환.
boxes, scores, classes, valid_len = export_model.f(img_tensor)
# Tensor값을 시각화를 위해 numpy 로 변환.
boxes = boxes.numpy()
scores = scores.numpy()
classes = classes.numpy()
valid_len = valid_len.numpy()
# detected 된 object들을 iteration 하면서 정보 추출. detect된 object의 갯수는 100개
for i in range(valid_len[0]):
# detection score를 iteration시 마다 높은 순으로 추출하고 SCORE_THRESHOLD보다 낮으면 loop 중단.
score = scores[0, i]
# detected된 object들은 scale된 기준으로 예측되었으므로 다시 원본 이미지 비율로 계산
box = boxes[0, i]
''' **** 주의 ******
box는 ymin, xmin, ymax, xmax 순서로 되어 있음. 또한 원본 좌표값으로 되어 있음. '''
left = box[1]
top = box[0]
right = box[3]
bottom = box[2]
# class id 추출하고 class 명으로 매핑
class_id = classes[0, i]
caption = "{}: {:.4f}".format(labels_to_names[class_id], score)
print(caption)
#cv2.rectangle()은 인자로 들어온 draw_img에 사각형을 그림. 위치 인자는 반드시 정수형.
cv2.rectangle(draw_img, (int(left), int(top)), (int(right), int(bottom)), color=green_color, thickness=2)
cv2.putText(draw_img, caption, (int(left), int(top - 5)), cv2.FONT_HERSHEY_SIMPLEX, 0.4, red_color, 1)
if is_print:
print('Detection 수행시간:',round(time.time() - start_time, 2),"초")
return draw_img
import cv2
img_array = cv2.cvtColor(cv2.imread('/content/data/beatles01.jpg'), cv2.COLOR_BGR2RGB)
draw_img = get_detected_img(export_model, img_array, is_print=True)
plt.figure(figsize=(16, 16))
plt.imshow(draw_img)
person: 0.9398
person: 0.9376
person: 0.8987
person: 0.8970
car: 0.7241
car: 0.5073
Detection 수행시간: 0.02 초
반응형
'Computer_Science > RetinaNet' 카테고리의 다른 글
| 12-7. AutoML efficientDet으로 esri object detection (0) | 2021.12.10 |
|---|---|
| 12-6. autoML EfficientDet - tfrecords 이해 (0) | 2021.12.10 |
| 12-1. autoML efficientDet custom 데이터 train (0) | 2021.12.08 |
| 11-9. 이미지 크기 변경 후 efficient det inference (0) | 2021.12.08 |
| 11-6~8. automl efficientDet (0) | 2021.11.30 |