Book a Demo!
CoCalc Logo Icon
StoreFeaturesDocsShareSupportNewsAboutPoliciesSign UpSign In
keras-team
GitHub Repository: keras-team/keras-io
Path: blob/master/examples/vision/md/basnet_segmentation.md
8010 views

Highly accurate boundaries segmentation using BASNet

Author: Hamid Ali
Date created: 2023/05/30
Last modified: 2025/01/24
Description: Boundaries aware segmentation model trained on the DUTS dataset.

View in Colab GitHub source


Introduction

Deep semantic segmentation algorithms have improved a lot recently, but still fails to correctly predict pixels around object boundaries. In this example we implement Boundary-Aware Segmentation Network (BASNet), using two stage predict and refine architecture, and a hybrid loss it can predict highly accurate boundaries and fine structures for image segmentation.

References:


Download the Data

We will use the DUTS-TE dataset for training. It has 5,019 images but we will use 140 for training and validation to save notebook running time. DUTS is relatively large salient object segmentation dataset. which contain diversified textures and structures common to real-world images in both foreground and background.

import os # Because of the use of tf.image.ssim in the loss, # this example requires TensorFlow. The rest of the code # is backend-agnostic. os.environ["KERAS_BACKEND"] = "tensorflow" import numpy as np from glob import glob import matplotlib.pyplot as plt import keras_hub import tensorflow as tf import keras from keras import layers, ops keras.config.disable_traceback_filtering()

Define Hyperparameters

IMAGE_SIZE = 288 BATCH_SIZE = 4 OUT_CLASSES = 1 TRAIN_SPLIT_RATIO = 0.90

Create PyDatasets

We will use load_paths() to load and split 140 paths into train and validation set, and convert paths into PyDataset object.

data_dir = keras.utils.get_file( origin="http://saliencydetection.net/duts/download/DUTS-TE.zip", extract=True, ) data_dir = os.path.join(data_dir, "DUTS-TE") def load_paths(path, split_ratio): images = sorted(glob(os.path.join(path, "DUTS-TE-Image/*")))[:140] masks = sorted(glob(os.path.join(path, "DUTS-TE-Mask/*")))[:140] len_ = int(len(images) * split_ratio) return (images[:len_], masks[:len_]), (images[len_:], masks[len_:]) class Dataset(keras.utils.PyDataset): def __init__( self, image_paths, mask_paths, img_size, out_classes, batch, shuffle=True, **kwargs, ): if shuffle: perm = np.random.permutation(len(image_paths)) image_paths = [image_paths[i] for i in perm] mask_paths = [mask_paths[i] for i in perm] self.image_paths = image_paths self.mask_paths = mask_paths self.img_size = img_size self.out_classes = out_classes self.batch_size = batch super().__init__(*kwargs) def __len__(self): return len(self.image_paths) // self.batch_size def __getitem__(self, idx): batch_x, batch_y = [], [] for i in range(idx * self.batch_size, (idx + 1) * self.batch_size): x, y = self.preprocess( self.image_paths[i], self.mask_paths[i], self.img_size, ) batch_x.append(x) batch_y.append(y) batch_x = np.stack(batch_x, axis=0) batch_y = np.stack(batch_y, axis=0) return batch_x, batch_y def read_image(self, path, size, mode): x = keras.utils.load_img(path, target_size=size, color_mode=mode) x = keras.utils.img_to_array(x) x = (x / 255.0).astype(np.float32) return x def preprocess(self, x_batch, y_batch, img_size): images = self.read_image(x_batch, (img_size, img_size), mode="rgb") # image masks = self.read_image(y_batch, (img_size, img_size), mode="grayscale") # mask return images, masks train_paths, val_paths = load_paths(data_dir, TRAIN_SPLIT_RATIO) train_dataset = Dataset( train_paths[0], train_paths[1], IMAGE_SIZE, OUT_CLASSES, BATCH_SIZE, shuffle=True ) val_dataset = Dataset( val_paths[0], val_paths[1], IMAGE_SIZE, OUT_CLASSES, BATCH_SIZE, shuffle=False )

Visualize Data

def display(display_list): title = ["Input Image", "True Mask", "Predicted Mask"] for i in range(len(display_list)): plt.subplot(1, len(display_list), i + 1) plt.title(title[i]) plt.imshow(keras.utils.array_to_img(display_list[i]), cmap="gray") plt.axis("off") plt.show() for image, mask in val_dataset: display([image[0], mask[0]]) break

png


Analyze Mask

Lets print unique values of above displayed mask. You can see despite belonging to one class, it's intensity is changing between low(0) to high(255). This variation in intensity makes it hard for network to generate good segmentation map for salient or camouflaged object segmentation. Because of its Residual Refined Module (RMs), BASNet is good in generating highly accurate boundaries and fine structures.

print(f"Unique values count: {len(np.unique((mask[0] * 255)))}") print("Unique values:") print(np.unique((mask[0] * 255)).astype(int))
``` Unique values count: 245 Unique values: [ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 61 62 63 65 66 67 68 69 70 71 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 108 109 110 111 112 113 114 115 116 117 118 119 120 122 123 124 125 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255]
</div> --- ## Building the BASNet Model BASNet comprises of a predict-refine architecture and a hybrid loss. The predict-refine architecture consists of a densely supervised encoder-decoder network and a residual refinement module, which are respectively used to predict and refine a segmentation probability map. ![](https://i.imgur.com/8jaZ2qs.png) ```python def basic_block(x_input, filters, stride=1, down_sample=None, activation=None): """Creates a residual(identity) block with two 3*3 convolutions.""" residual = x_input x = layers.Conv2D(filters, (3, 3), strides=stride, padding="same", use_bias=False)( x_input ) x = layers.BatchNormalization()(x) x = layers.Activation("relu")(x) x = layers.Conv2D(filters, (3, 3), strides=(1, 1), padding="same", use_bias=False)( x ) x = layers.BatchNormalization()(x) if down_sample is not None: residual = down_sample x = layers.Add()([x, residual]) if activation is not None: x = layers.Activation(activation)(x) return x def convolution_block(x_input, filters, dilation=1): """Apply convolution + batch normalization + relu layer.""" x = layers.Conv2D(filters, (3, 3), padding="same", dilation_rate=dilation)(x_input) x = layers.BatchNormalization()(x) return layers.Activation("relu")(x) def segmentation_head(x_input, out_classes, final_size): """Map each decoder stage output to model output classes.""" x = layers.Conv2D(out_classes, kernel_size=(3, 3), padding="same")(x_input) if final_size is not None: x = layers.Resizing(final_size[0], final_size[1])(x) return x def get_resnet_block(resnet, block_num): """Extract and return a ResNet-34 block.""" extractor_levels = ["P2", "P3", "P4", "P5"] num_blocks = resnet.stackwise_num_blocks if block_num == 0: x = resnet.get_layer("pool1_pool").output else: x = resnet.pyramid_outputs[extractor_levels[block_num - 1]] y = resnet.get_layer(f"stack{block_num}_block{num_blocks[block_num]-1}_add").output return keras.models.Model( inputs=x, outputs=y, name=f"resnet_block{block_num + 1}", )

Prediction Module

Prediction module is a heavy encoder decoder structure like U-Net. The encoder includes an input convolutional layer and six stages. First four are adopted from ResNet-34 and rest are basic res-blocks. Since first convolution and pooling layer of ResNet-34 is skipped so we will use get_resnet_block() to extract first four blocks. Both bridge and decoder uses three convolutional layers with side outputs. The module produces seven segmentation probability maps during training, with the last one considered the final output.

def basnet_predict(input_shape, out_classes): """BASNet Prediction Module, it outputs coarse label map.""" filters = 64 num_stages = 6 x_input = layers.Input(input_shape) # -------------Encoder-------------- x = layers.Conv2D(filters, kernel_size=(3, 3), padding="same")(x_input) resnet = keras_hub.models.ResNetBackbone( input_conv_filters=[64], input_conv_kernel_sizes=[7], stackwise_num_filters=[64, 128, 256, 512], stackwise_num_blocks=[3, 4, 6, 3], stackwise_num_strides=[1, 2, 2, 2], block_type="basic_block", ) encoder_blocks = [] for i in range(num_stages): if i < 4: # First four stages are adopted from ResNet-34 blocks. x = get_resnet_block(resnet, i)(x) encoder_blocks.append(x) x = layers.Activation("relu")(x) else: # Last 2 stages consist of three basic resnet blocks. x = layers.MaxPool2D(pool_size=(2, 2), strides=(2, 2))(x) x = basic_block(x, filters=filters * 8, activation="relu") x = basic_block(x, filters=filters * 8, activation="relu") x = basic_block(x, filters=filters * 8, activation="relu") encoder_blocks.append(x) # -------------Bridge------------- x = convolution_block(x, filters=filters * 8, dilation=2) x = convolution_block(x, filters=filters * 8, dilation=2) x = convolution_block(x, filters=filters * 8, dilation=2) encoder_blocks.append(x) # -------------Decoder------------- decoder_blocks = [] for i in reversed(range(num_stages)): if i != (num_stages - 1): # Except first, scale other decoder stages. shape = x.shape x = layers.Resizing(shape[1] * 2, shape[2] * 2)(x) x = layers.concatenate([encoder_blocks[i], x], axis=-1) x = convolution_block(x, filters=filters * 8) x = convolution_block(x, filters=filters * 8) x = convolution_block(x, filters=filters * 8) decoder_blocks.append(x) decoder_blocks.reverse() # Change order from last to first decoder stage. decoder_blocks.append(encoder_blocks[-1]) # Copy bridge to decoder. # -------------Side Outputs-------------- decoder_blocks = [ segmentation_head(decoder_block, out_classes, input_shape[:2]) for decoder_block in decoder_blocks ] return keras.models.Model(inputs=x_input, outputs=decoder_blocks)

Residual Refinement Module

Refinement Modules (RMs), designed as a residual block aim to refines the coarse(blurry and noisy boundaries) segmentation maps generated by prediction module. Similar to prediction module it's also an encode decoder structure but with light weight 4 stages, each containing one convolutional block() init. At the end it adds both coarse and residual output to generate refined output.

def basnet_rrm(base_model, out_classes): """BASNet Residual Refinement Module(RRM) module, output fine label map.""" num_stages = 4 filters = 64 x_input = base_model.output[0] # -------------Encoder-------------- x = layers.Conv2D(filters, kernel_size=(3, 3), padding="same")(x_input) encoder_blocks = [] for _ in range(num_stages): x = convolution_block(x, filters=filters) encoder_blocks.append(x) x = layers.MaxPool2D(pool_size=(2, 2), strides=(2, 2))(x) # -------------Bridge-------------- x = convolution_block(x, filters=filters) # -------------Decoder-------------- for i in reversed(range(num_stages)): shape = x.shape x = layers.Resizing(shape[1] * 2, shape[2] * 2)(x) x = layers.concatenate([encoder_blocks[i], x], axis=-1) x = convolution_block(x, filters=filters) x = segmentation_head(x, out_classes, None) # Segmentation head. # ------------- refined = coarse + residual x = layers.Add()([x_input, x]) # Add prediction + refinement output return keras.models.Model(inputs=[base_model.input], outputs=[x])

Combine Predict and Refinement Module

class BASNet(keras.Model): def __init__(self, input_shape, out_classes): """BASNet, it's a combination of two modules Prediction Module and Residual Refinement Module(RRM).""" # Prediction model. predict_model = basnet_predict(input_shape, out_classes) # Refinement model. refine_model = basnet_rrm(predict_model, out_classes) output = refine_model.outputs # Combine outputs. output.extend(predict_model.output) # Activations. output = [layers.Activation("sigmoid")(x) for x in output] super().__init__(inputs=predict_model.input, outputs=output) self.smooth = 1.0e-9 # Binary Cross Entropy loss. self.cross_entropy_loss = keras.losses.BinaryCrossentropy() # Structural Similarity Index value. self.ssim_value = tf.image.ssim # Jaccard / IoU loss. self.iou_value = self.calculate_iou def calculate_iou( self, y_true, y_pred, ): """Calculate intersection over union (IoU) between images.""" intersection = ops.sum(ops.abs(y_true * y_pred), axis=[1, 2, 3]) union = ops.sum(y_true, [1, 2, 3]) + ops.sum(y_pred, [1, 2, 3]) union = union - intersection return ops.mean((intersection + self.smooth) / (union + self.smooth), axis=0) def compute_loss(self, x, y_true, y_pred, sample_weight=None, training=False): total = 0.0 for y_pred_i in y_pred: # y_pred = refine_model.outputs + predict_model.output cross_entropy_loss = self.cross_entropy_loss(y_true, y_pred_i) ssim_value = self.ssim_value(y_true, y_pred, max_val=1) ssim_loss = ops.mean(1 - ssim_value + self.smooth, axis=0) iou_value = self.iou_value(y_true, y_pred) iou_loss = 1 - iou_value # Add all three losses. total += cross_entropy_loss + ssim_loss + iou_loss return total

Hybrid Loss

Another important feature of BASNet is its hybrid loss function, which is a combination of binary cross entropy, structural similarity and intersection-over-union losses, which guide the network to learn three-level (i.e., pixel, patch and map level) hierarchy representations.

basnet_model = BASNet( input_shape=[IMAGE_SIZE, IMAGE_SIZE, 3], out_classes=OUT_CLASSES ) # Create model. basnet_model.summary() # Show model summary. optimizer = keras.optimizers.Adam(learning_rate=1e-4, epsilon=1e-8) # Compile model. basnet_model.compile( optimizer=optimizer, metrics=[keras.metrics.MeanAbsoluteError(name="mae") for _ in basnet_model.outputs], )
Model: "bas_net"
┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓
┃ Layer (type)         Output Shape          Param #  Connected to      ┃
┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩
│ input_layer         │ (None, 288, 288,  │          0 │ -                 │
│ (InputLayer)        │ 3)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d (Conv2D)     │ (None, 288, 288,  │      1,792 │ input_layer[0][0] │
│                     │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resnet_block1       │ (None, 288, 288,  │    222,720 │ conv2d[0][0]      │
│ (Functional)        │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation          │ (None, 288, 288,  │          0 │ resnet_block1[0]… │
│ (Activation)        │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resnet_block2       │ (None, 144, 144,  │  1,118,720 │ activation[0][0]  │
│ (Functional)        │ 128)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_1        │ (None, 144, 144,  │          0 │ resnet_block2[0]… │
│ (Activation)        │ 128)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resnet_block3       │ (None, 72, 72,    │  6,829,056 │ activation_1[0][ │
│ (Functional)        │ 256)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_2        │ (None, 72, 72,    │          0 │ resnet_block3[0]… │
│ (Activation)        │ 256)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resnet_block4       │ (None, 36, 36,    │ 13,121,536 │ activation_2[0][ │
│ (Functional)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_3        │ (None, 36, 36,    │          0 │ resnet_block4[0]… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ max_pooling2d       │ (None, 18, 18,    │          0 │ activation_3[0][ │
│ (MaxPooling2D)      │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_1 (Conv2D)   │ (None, 18, 18,    │  2,359,296 │ max_pooling2d[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalization │ (None, 18, 18,    │      2,048 │ conv2d_1[0][0]    │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_4        │ (None, 18, 18,    │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_2 (Conv2D)   │ (None, 18, 18,    │  2,359,296 │ activation_4[0][ │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 18, 18,    │      2,048 │ conv2d_2[0][0]    │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ add (Add)           │ (None, 18, 18,    │          0 │ batch_normalizat… │
│                     │ 512)              │            │ max_pooling2d[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_5        │ (None, 18, 18,    │          0 │ add[0][0]         │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_3 (Conv2D)   │ (None, 18, 18,    │  2,359,296 │ activation_5[0][ │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 18, 18,    │      2,048 │ conv2d_3[0][0]    │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_6        │ (None, 18, 18,    │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_4 (Conv2D)   │ (None, 18, 18,    │  2,359,296 │ activation_6[0][ │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 18, 18,    │      2,048 │ conv2d_4[0][0]    │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ add_1 (Add)         │ (None, 18, 18,    │          0 │ batch_normalizat… │
│                     │ 512)              │            │ activation_5[0][ │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_7        │ (None, 18, 18,    │          0 │ add_1[0][0]       │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_5 (Conv2D)   │ (None, 18, 18,    │  2,359,296 │ activation_7[0][ │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 18, 18,    │      2,048 │ conv2d_5[0][0]    │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_8        │ (None, 18, 18,    │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_6 (Conv2D)   │ (None, 18, 18,    │  2,359,296 │ activation_8[0][ │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 18, 18,    │      2,048 │ conv2d_6[0][0]    │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ add_2 (Add)         │ (None, 18, 18,    │          0 │ batch_normalizat… │
│                     │ 512)              │            │ activation_7[0][ │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_9        │ (None, 18, 18,    │          0 │ add_2[0][0]       │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ max_pooling2d_1     │ (None, 9, 9, 512) │          0 │ activation_9[0][ │
│ (MaxPooling2D)      │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_7 (Conv2D)   │ (None, 9, 9, 512) │  2,359,296 │ max_pooling2d_1[ │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 9, 9, 512) │      2,048 │ conv2d_7[0][0]    │
│ (BatchNormalizatio… │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_10       │ (None, 9, 9, 512) │          0 │ batch_normalizat… │
│ (Activation)        │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_8 (Conv2D)   │ (None, 9, 9, 512) │  2,359,296 │ activation_10[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 9, 9, 512) │      2,048 │ conv2d_8[0][0]    │
│ (BatchNormalizatio… │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ add_3 (Add)         │ (None, 9, 9, 512) │          0 │ batch_normalizat… │
│                     │                   │            │ max_pooling2d_1[ │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_11       │ (None, 9, 9, 512) │          0 │ add_3[0][0]       │
│ (Activation)        │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_9 (Conv2D)   │ (None, 9, 9, 512) │  2,359,296 │ activation_11[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 9, 9, 512) │      2,048 │ conv2d_9[0][0]    │
│ (BatchNormalizatio… │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_12       │ (None, 9, 9, 512) │          0 │ batch_normalizat… │
│ (Activation)        │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_10 (Conv2D)  │ (None, 9, 9, 512) │  2,359,296 │ activation_12[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 9, 9, 512) │      2,048 │ conv2d_10[0][0]   │
│ (BatchNormalizatio… │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ add_4 (Add)         │ (None, 9, 9, 512) │          0 │ batch_normalizat… │
│                     │                   │            │ activation_11[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_13       │ (None, 9, 9, 512) │          0 │ add_4[0][0]       │
│ (Activation)        │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_11 (Conv2D)  │ (None, 9, 9, 512) │  2,359,296 │ activation_13[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 9, 9, 512) │      2,048 │ conv2d_11[0][0]   │
│ (BatchNormalizatio… │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_14       │ (None, 9, 9, 512) │          0 │ batch_normalizat… │
│ (Activation)        │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_12 (Conv2D)  │ (None, 9, 9, 512) │  2,359,296 │ activation_14[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 9, 9, 512) │      2,048 │ conv2d_12[0][0]   │
│ (BatchNormalizatio… │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ add_5 (Add)         │ (None, 9, 9, 512) │          0 │ batch_normalizat… │
│                     │                   │            │ activation_13[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_15       │ (None, 9, 9, 512) │          0 │ add_5[0][0]       │
│ (Activation)        │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_13 (Conv2D)  │ (None, 9, 9, 512) │  2,359,808 │ activation_15[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 9, 9, 512) │      2,048 │ conv2d_13[0][0]   │
│ (BatchNormalizatio… │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_16       │ (None, 9, 9, 512) │          0 │ batch_normalizat… │
│ (Activation)        │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_14 (Conv2D)  │ (None, 9, 9, 512) │  2,359,808 │ activation_16[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 9, 9, 512) │      2,048 │ conv2d_14[0][0]   │
│ (BatchNormalizatio… │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_17       │ (None, 9, 9, 512) │          0 │ batch_normalizat… │
│ (Activation)        │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_15 (Conv2D)  │ (None, 9, 9, 512) │  2,359,808 │ activation_17[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 9, 9, 512) │      2,048 │ conv2d_15[0][0]   │
│ (BatchNormalizatio… │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_18       │ (None, 9, 9, 512) │          0 │ batch_normalizat… │
│ (Activation)        │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ concatenate         │ (None, 9, 9,      │          0 │ activation_15[0]… │
│ (Concatenate)       │ 1024)             │            │ activation_18[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_16 (Conv2D)  │ (None, 9, 9, 512) │  4,719,104 │ concatenate[0][0] │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 9, 9, 512) │      2,048 │ conv2d_16[0][0]   │
│ (BatchNormalizatio… │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_19       │ (None, 9, 9, 512) │          0 │ batch_normalizat… │
│ (Activation)        │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_17 (Conv2D)  │ (None, 9, 9, 512) │  2,359,808 │ activation_19[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 9, 9, 512) │      2,048 │ conv2d_17[0][0]   │
│ (BatchNormalizatio… │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_20       │ (None, 9, 9, 512) │          0 │ batch_normalizat… │
│ (Activation)        │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_18 (Conv2D)  │ (None, 9, 9, 512) │  2,359,808 │ activation_20[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 9, 9, 512) │      2,048 │ conv2d_18[0][0]   │
│ (BatchNormalizatio… │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_21       │ (None, 9, 9, 512) │          0 │ batch_normalizat… │
│ (Activation)        │                   │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing (Resizing) │ (None, 18, 18,    │          0 │ activation_21[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ concatenate_1       │ (None, 18, 18,    │          0 │ activation_9[0][ │
│ (Concatenate)       │ 1024)             │            │ resizing[0][0]    │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_19 (Conv2D)  │ (None, 18, 18,    │  4,719,104 │ concatenate_1[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 18, 18,    │      2,048 │ conv2d_19[0][0]   │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_22       │ (None, 18, 18,    │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_20 (Conv2D)  │ (None, 18, 18,    │  2,359,808 │ activation_22[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 18, 18,    │      2,048 │ conv2d_20[0][0]   │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_23       │ (None, 18, 18,    │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_21 (Conv2D)  │ (None, 18, 18,    │  2,359,808 │ activation_23[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 18, 18,    │      2,048 │ conv2d_21[0][0]   │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_24       │ (None, 18, 18,    │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing_1          │ (None, 36, 36,    │          0 │ activation_24[0]… │
│ (Resizing)          │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ concatenate_2       │ (None, 36, 36,    │          0 │ resnet_block4[0]… │
│ (Concatenate)       │ 1024)             │            │ resizing_1[0][0]  │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_22 (Conv2D)  │ (None, 36, 36,    │  4,719,104 │ concatenate_2[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 36, 36,    │      2,048 │ conv2d_22[0][0]   │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_25       │ (None, 36, 36,    │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_23 (Conv2D)  │ (None, 36, 36,    │  2,359,808 │ activation_25[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 36, 36,    │      2,048 │ conv2d_23[0][0]   │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_26       │ (None, 36, 36,    │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_24 (Conv2D)  │ (None, 36, 36,    │  2,359,808 │ activation_26[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 36, 36,    │      2,048 │ conv2d_24[0][0]   │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_27       │ (None, 36, 36,    │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing_2          │ (None, 72, 72,    │          0 │ activation_27[0]… │
│ (Resizing)          │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ concatenate_3       │ (None, 72, 72,    │          0 │ resnet_block3[0]… │
│ (Concatenate)       │ 768)              │            │ resizing_2[0][0]  │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_25 (Conv2D)  │ (None, 72, 72,    │  3,539,456 │ concatenate_3[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 72, 72,    │      2,048 │ conv2d_25[0][0]   │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_28       │ (None, 72, 72,    │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_26 (Conv2D)  │ (None, 72, 72,    │  2,359,808 │ activation_28[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 72, 72,    │      2,048 │ conv2d_26[0][0]   │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_29       │ (None, 72, 72,    │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_27 (Conv2D)  │ (None, 72, 72,    │  2,359,808 │ activation_29[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 72, 72,    │      2,048 │ conv2d_27[0][0]   │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_30       │ (None, 72, 72,    │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing_3          │ (None, 144, 144,  │          0 │ activation_30[0]… │
│ (Resizing)          │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ concatenate_4       │ (None, 144, 144,  │          0 │ resnet_block2[0]… │
│ (Concatenate)       │ 640)              │            │ resizing_3[0][0]  │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_28 (Conv2D)  │ (None, 144, 144,  │  2,949,632 │ concatenate_4[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 144, 144,  │      2,048 │ conv2d_28[0][0]   │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_31       │ (None, 144, 144,  │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_29 (Conv2D)  │ (None, 144, 144,  │  2,359,808 │ activation_31[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 144, 144,  │      2,048 │ conv2d_29[0][0]   │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_32       │ (None, 144, 144,  │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_30 (Conv2D)  │ (None, 144, 144,  │  2,359,808 │ activation_32[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 144, 144,  │      2,048 │ conv2d_30[0][0]   │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_33       │ (None, 144, 144,  │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing_4          │ (None, 288, 288,  │          0 │ activation_33[0]… │
│ (Resizing)          │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ concatenate_5       │ (None, 288, 288,  │          0 │ resnet_block1[0]… │
│ (Concatenate)       │ 576)              │            │ resizing_4[0][0]  │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_31 (Conv2D)  │ (None, 288, 288,  │  2,654,720 │ concatenate_5[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 288, 288,  │      2,048 │ conv2d_31[0][0]   │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_34       │ (None, 288, 288,  │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_32 (Conv2D)  │ (None, 288, 288,  │  2,359,808 │ activation_34[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 288, 288,  │      2,048 │ conv2d_32[0][0]   │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_35       │ (None, 288, 288,  │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_33 (Conv2D)  │ (None, 288, 288,  │  2,359,808 │ activation_35[0]… │
│                     │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 288, 288,  │      2,048 │ conv2d_33[0][0]   │
│ (BatchNormalizatio…512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_36       │ (None, 288, 288,  │          0 │ batch_normalizat… │
│ (Activation)        │ 512)              │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_34 (Conv2D)  │ (None, 288, 288,  │      4,609 │ activation_36[0]… │
│                     │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing_5          │ (None, 288, 288,  │          0 │ conv2d_34[0][0]   │
│ (Resizing)          │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_41 (Conv2D)  │ (None, 288, 288,  │        640 │ resizing_5[0][0]  │
│                     │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_42 (Conv2D)  │ (None, 288, 288,  │     36,928 │ conv2d_41[0][0]   │
│                     │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 288, 288,  │        256 │ conv2d_42[0][0]   │
│ (BatchNormalizatio…64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_37       │ (None, 288, 288,  │          0 │ batch_normalizat… │
│ (Activation)        │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ max_pooling2d_2     │ (None, 144, 144,  │          0 │ activation_37[0]… │
│ (MaxPooling2D)      │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_43 (Conv2D)  │ (None, 144, 144,  │     36,928 │ max_pooling2d_2[ │
│                     │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 144, 144,  │        256 │ conv2d_43[0][0]   │
│ (BatchNormalizatio…64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_38       │ (None, 144, 144,  │          0 │ batch_normalizat… │
│ (Activation)        │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ max_pooling2d_3     │ (None, 72, 72,    │          0 │ activation_38[0]… │
│ (MaxPooling2D)      │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_44 (Conv2D)  │ (None, 72, 72,    │     36,928 │ max_pooling2d_3[ │
│                     │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 72, 72,    │        256 │ conv2d_44[0][0]   │
│ (BatchNormalizatio…64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_39       │ (None, 72, 72,    │          0 │ batch_normalizat… │
│ (Activation)        │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ max_pooling2d_4     │ (None, 36, 36,    │          0 │ activation_39[0]… │
│ (MaxPooling2D)      │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_45 (Conv2D)  │ (None, 36, 36,    │     36,928 │ max_pooling2d_4[ │
│                     │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 36, 36,    │        256 │ conv2d_45[0][0]   │
│ (BatchNormalizatio…64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_40       │ (None, 36, 36,    │          0 │ batch_normalizat… │
│ (Activation)        │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ max_pooling2d_5     │ (None, 18, 18,    │          0 │ activation_40[0]… │
│ (MaxPooling2D)      │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_46 (Conv2D)  │ (None, 18, 18,    │     36,928 │ max_pooling2d_5[ │
│                     │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 18, 18,    │        256 │ conv2d_46[0][0]   │
│ (BatchNormalizatio…64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_41       │ (None, 18, 18,    │          0 │ batch_normalizat… │
│ (Activation)        │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing_12         │ (None, 36, 36,    │          0 │ activation_41[0]… │
│ (Resizing)          │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ concatenate_6       │ (None, 36, 36,    │          0 │ activation_40[0]… │
│ (Concatenate)       │ 128)              │            │ resizing_12[0][0] │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_47 (Conv2D)  │ (None, 36, 36,    │     73,792 │ concatenate_6[0]… │
│                     │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 36, 36,    │        256 │ conv2d_47[0][0]   │
│ (BatchNormalizatio…64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_42       │ (None, 36, 36,    │          0 │ batch_normalizat… │
│ (Activation)        │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing_13         │ (None, 72, 72,    │          0 │ activation_42[0]… │
│ (Resizing)          │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ concatenate_7       │ (None, 72, 72,    │          0 │ activation_39[0]… │
│ (Concatenate)       │ 128)              │            │ resizing_13[0][0] │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_48 (Conv2D)  │ (None, 72, 72,    │     73,792 │ concatenate_7[0]… │
│                     │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 72, 72,    │        256 │ conv2d_48[0][0]   │
│ (BatchNormalizatio…64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_43       │ (None, 72, 72,    │          0 │ batch_normalizat… │
│ (Activation)        │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing_14         │ (None, 144, 144,  │          0 │ activation_43[0]… │
│ (Resizing)          │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ concatenate_8       │ (None, 144, 144,  │          0 │ activation_38[0]… │
│ (Concatenate)       │ 128)              │            │ resizing_14[0][0] │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_49 (Conv2D)  │ (None, 144, 144,  │     73,792 │ concatenate_8[0]… │
│                     │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 144, 144,  │        256 │ conv2d_49[0][0]   │
│ (BatchNormalizatio…64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_44       │ (None, 144, 144,  │          0 │ batch_normalizat… │
│ (Activation)        │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing_15         │ (None, 288, 288,  │          0 │ activation_44[0]… │
│ (Resizing)          │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ concatenate_9       │ (None, 288, 288,  │          0 │ activation_37[0]… │
│ (Concatenate)       │ 128)              │            │ resizing_15[0][0] │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_50 (Conv2D)  │ (None, 288, 288,  │     73,792 │ concatenate_9[0]… │
│                     │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ batch_normalizatio… │ (None, 288, 288,  │        256 │ conv2d_50[0][0]   │
│ (BatchNormalizatio…64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_45       │ (None, 288, 288,  │          0 │ batch_normalizat… │
│ (Activation)        │ 64)               │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_51 (Conv2D)  │ (None, 288, 288,  │        577 │ activation_45[0]… │
│                     │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_35 (Conv2D)  │ (None, 144, 144,  │      4,609 │ activation_33[0]… │
│                     │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_36 (Conv2D)  │ (None, 72, 72, 1) │      4,609 │ activation_30[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_37 (Conv2D)  │ (None, 36, 36, 1) │      4,609 │ activation_27[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_38 (Conv2D)  │ (None, 18, 18, 1) │      4,609 │ activation_24[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_39 (Conv2D)  │ (None, 9, 9, 1)   │      4,609 │ activation_21[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ conv2d_40 (Conv2D)  │ (None, 9, 9, 1)   │      4,609 │ activation_18[0]… │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ add_6 (Add)         │ (None, 288, 288,  │          0 │ resizing_5[0][0], │
│                     │ 1)                │            │ conv2d_51[0][0]   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing_6          │ (None, 288, 288,  │          0 │ conv2d_35[0][0]   │
│ (Resizing)          │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing_7          │ (None, 288, 288,  │          0 │ conv2d_36[0][0]   │
│ (Resizing)          │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing_8          │ (None, 288, 288,  │          0 │ conv2d_37[0][0]   │
│ (Resizing)          │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing_9          │ (None, 288, 288,  │          0 │ conv2d_38[0][0]   │
│ (Resizing)          │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing_10         │ (None, 288, 288,  │          0 │ conv2d_39[0][0]   │
│ (Resizing)          │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ resizing_11         │ (None, 288, 288,  │          0 │ conv2d_40[0][0]   │
│ (Resizing)          │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_46       │ (None, 288, 288,  │          0 │ add_6[0][0]       │
│ (Activation)        │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_47       │ (None, 288, 288,  │          0 │ resizing_5[0][0]  │
│ (Activation)        │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_48       │ (None, 288, 288,  │          0 │ resizing_6[0][0]  │
│ (Activation)        │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_49       │ (None, 288, 288,  │          0 │ resizing_7[0][0]  │
│ (Activation)        │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_50       │ (None, 288, 288,  │          0 │ resizing_8[0][0]  │
│ (Activation)        │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_51       │ (None, 288, 288,  │          0 │ resizing_9[0][0]  │
│ (Activation)        │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_52       │ (None, 288, 288,  │          0 │ resizing_10[0][0] │
│ (Activation)        │ 1)                │            │                   │
├─────────────────────┼───────────────────┼────────────┼───────────────────┤
│ activation_53       │ (None, 288, 288,  │          0 │ resizing_11[0][0] │
│ (Activation)        │ 1)                │            │                   │
└─────────────────────┴───────────────────┴────────────┴───────────────────┘
 Total params: 108,886,792 (415.37 MB)
 Trainable params: 108,834,952 (415.17 MB)
 Non-trainable params: 51,840 (202.50 KB)

Train the Model

basnet_model.fit(train_dataset, validation_data=val_dataset, epochs=1)
``` 31/31 ━━━━━━━━━━━━━━━━━━━━ 38319s 1245s/step - activation_46_mae: 0.2864 - loss: 17.4035 - val_activation_46_mae: 0.8427 - val_loss: 238.6499

<keras.src.callbacks.history.History at 0x1312ff970>

</div> ### Visualize Predictions In paper BASNet was trained on DUTS-TR dataset, which has 10553 images. Model was trained for 400k iterations with a batch size of eight and without a validation dataset. After training model was evaluated on DUTS-TE dataset and achieved a mean absolute error of `0.042`. Since BASNet is a deep model and cannot be trained in a short amount of time which is a requirement for keras example notebook, so we will load pretrained weights from [here](https://github.com/hamidriasat/BASNet/tree/basnet_keras) to show model prediction. Due to computer power limitation this model was trained for 120k iterations but it still demonstrates its capabilities. For further details about trainings parameters please check given link. ```python import gdown gdown.download(id="1OWKouuAQ7XpXZbWA3mmxDPrFGW71Axrg", output="basnet_weights.h5") def normalize_output(prediction): max_value = np.max(prediction) min_value = np.min(prediction) return (prediction - min_value) / (max_value - min_value) # Load weights. basnet_model.load_weights("./basnet_weights.h5")
``` Downloading... From (original): https://drive.google.com/uc?id=1OWKouuAQ7XpXZbWA3mmxDPrFGW71Axrg From (redirected): https://drive.google.com/uc?id=1OWKouuAQ7XpXZbWA3mmxDPrFGW71Axrg&confirm=t&uuid=57f729b1-764a-4fbb-a569-c6eb5dd414ef To: /Users/laxmareddyp/Desktop/Keras-IO/keras-io/scripts/tmp_4610020/basnet_weights.h5

Python(47905) MallocStackLogging: can't turn off malloc stack logging because it was not enabled.

</div> 0%| | 0.00/436M [00:00<?, ?B/s] 0%| | 1.05M/436M [00:00<00:42, 10.2MB/s] 0%| | 2.10M/436M [00:00<00:42, 10.1MB/s] 1%| | 3.67M/436M [00:00<00:40, 10.7MB/s] 1%| | 5.24M/436M [00:00<00:37, 11.5MB/s] 2%| | 6.82M/436M [00:00<00:36, 11.8MB/s] 2%| | 8.39M/436M [00:00<00:33, 12.8MB/s] 2%| | 9.96M/436M [00:00<00:31, 13.5MB/s] 3%| | 12.1M/436M [00:00<00:28, 14.9MB/s] 3%| | 14.7M/436M [00:01<00:24, 17.0MB/s] 4%| | 17.3M/436M [00:01<00:22, 18.2MB/s] 5%| | 19.9M/436M [00:01<00:21, 19.6MB/s] 5%| | 22.5M/436M [00:01<00:20, 20.3MB/s] 6%| | 25.2M/436M [00:01<00:19, 20.7MB/s] 6%| | 27.3M/436M [00:01<00:20, 20.3MB/s] 7%| | 29.9M/436M [00:01<00:19, 20.7MB/s] 7%| | 32.0M/436M [00:01<00:19, 20.5MB/s] 8%| | 34.6M/436M [00:01<00:19, 20.7MB/s] 8%| | 36.7M/436M [00:02<00:19, 20.7MB/s] 9%| | 38.8M/436M [00:02<00:19, 20.1MB/s] 9%| | 41.4M/436M [00:02<00:18, 21.4MB/s] 10%| | 44.0M/436M [00:02<00:18, 21.5MB/s] 11%| | 46.7M/436M [00:02<00:18, 21.0MB/s] 11%| | 49.3M/436M [00:02<00:19, 20.1MB/s] 12%| | 51.9M/436M [00:02<00:18, 21.2MB/s] 13%| | 54.5M/436M [00:02<00:18, 21.0MB/s] 13%| | 57.1M/436M [00:03<00:18, 20.8MB/s] 14%| | 59.2M/436M [00:03<00:18, 20.6MB/s] 14%| | 61.3M/436M [00:03<00:18, 19.8MB/s] 15%| | 63.4M/436M [00:03<00:19, 19.5MB/s] 15%| | 65.5M/436M [00:03<00:18, 19.7MB/s] 16%| | 68.2M/436M [00:03<00:18, 20.2MB/s] 16%| | 70.3M/436M [00:03<00:17, 20.4MB/s] 17%| | 72.4M/436M [00:03<00:18, 20.1MB/s] 17%| | 75.0M/436M [00:03<00:18, 20.0MB/s] 18%| | 77.6M/436M [00:04<00:17, 20.1MB/s] 18%| | 80.2M/436M [00:04<00:16, 21.1MB/s] 19%| | 82.8M/436M [00:04<00:16, 21.1MB/s] 20%| | 85.5M/436M [00:04<00:16, 21.4MB/s] 20%| | 88.1M/436M [00:04<00:16, 21.0MB/s] 21%| | 90.7M/436M [00:04<00:16, 21.1MB/s] 21%| | 93.3M/436M [00:04<00:16, 21.4MB/s] 22%| | 95.9M/436M [00:04<00:15, 21.5MB/s] 23%| | 98.6M/436M [00:05<00:15, 21.6MB/s] 23%| | 101M/436M [00:05<00:16, 20.1MB/s] 24%| | 103M/436M [00:05<00:16, 19.8MB/s] 24%| | 106M/436M [00:05<00:16, 20.2MB/s] 25%| | 108M/436M [00:05<00:22, 14.3MB/s] 25%| | 110M/436M [00:05<00:21, 15.3MB/s] 26%| | 112M/436M [00:05<00:19, 16.2MB/s] 26%| | 115M/436M [00:06<00:17, 18.2MB/s] 27%| | 117M/436M [00:06<00:16, 19.2MB/s] 28%| | 120M/436M [00:06<00:15, 20.0MB/s] 28%| | 123M/436M [00:06<00:15, 20.5MB/s] 29%| | 125M/436M [00:06<00:15, 20.7MB/s] 29%| | 128M/436M [00:06<00:14, 21.1MB/s] 30%| | 131M/436M [00:06<00:14, 20.7MB/s] 31%| | 133M/436M [00:06<00:14, 21.1MB/s] 31%| | 136M/436M [00:07<00:14, 21.2MB/s] 32%| | 138M/436M [00:07<00:13, 21.4MB/s] 32%| | 141M/436M [00:07<00:13, 21.5MB/s] 33%| | 144M/436M [00:07<00:13, 21.6MB/s] 34%| | 146M/436M [00:07<00:13, 21.6MB/s] 34%| | 149M/436M [00:07<00:13, 21.7MB/s] 35%| | 152M/436M [00:07<00:13, 21.2MB/s] 35%| | 154M/436M [00:07<00:13, 20.8MB/s] 36%| | 157M/436M [00:08<00:13, 20.9MB/s] 36%| | 159M/436M [00:08<00:14, 19.0MB/s] 37%| | 161M/436M [00:08<00:14, 19.5MB/s] 37%| | 163M/436M [00:08<00:14, 19.5MB/s] 38%| | 165M/436M [00:08<00:13, 19.6MB/s] 38%| | 167M/436M [00:08<00:13, 19.6MB/s] 39%| | 170M/436M [00:08<00:13, 20.2MB/s] 39%| | 172M/436M [00:08<00:13, 19.2MB/s] 40%| | 174M/436M [00:08<00:13, 19.3MB/s] 40%| | 176M/436M [00:09<00:13, 19.4MB/s] 41%| | 178M/436M [00:09<00:13, 19.6MB/s] 41%| | 180M/436M [00:09<00:13, 19.2MB/s] 42%| | 182M/436M [00:09<00:13, 19.1MB/s] 42%| | 185M/436M [00:09<00:13, 18.7MB/s] 43%| | 187M/436M [00:09<00:12, 19.2MB/s] 43%| | 189M/436M [00:09<00:12, 19.0MB/s] 44%| | 191M/436M [00:09<00:12, 18.9MB/s] 44%| | 193M/436M [00:09<00:13, 18.6MB/s] 45%| | 196M/436M [00:10<00:12, 18.6MB/s] 45%| | 198M/436M [00:10<00:12, 18.4MB/s] 46%| | 200M/436M [00:10<00:12, 19.5MB/s] 46%| | 202M/436M [00:10<00:11, 19.8MB/s] 47%| | 205M/436M [00:10<00:11, 20.4MB/s] 48%| | 208M/436M [00:10<00:10, 20.8MB/s] 48%| | 210M/436M [00:10<00:11, 19.2MB/s] 49%| | 213M/436M [00:10<00:10, 21.9MB/s] 49%| | 215M/436M [00:11<00:10, 21.8MB/s] 50%| | 218M/436M [00:11<00:10, 21.8MB/s] 51%| | 221M/436M [00:11<00:10, 21.4MB/s] 51%| | 223M/436M [00:11<00:10, 20.7MB/s] 52%| | 226M/436M [00:11<00:10, 20.9MB/s] 52%| | 229M/436M [00:11<00:09, 21.2MB/s] 53%| | 231M/436M [00:11<00:09, 21.0MB/s] 54%| | 234M/436M [00:11<00:09, 20.6MB/s] 54%| | 236M/436M [00:12<00:09, 20.1MB/s] 55%| | 238M/436M [00:12<00:09, 19.8MB/s] 55%| | 240M/436M [00:12<00:10, 19.6MB/s] 56%| | 242M/436M [00:12<00:09, 19.4MB/s] 56%| | 244M/436M [00:12<00:09, 19.2MB/s] 57%| | 246M/436M [00:12<00:09, 19.4MB/s] 57%| | 249M/436M [00:12<00:09, 19.5MB/s] 57%| | 251M/436M [00:12<00:09, 19.4MB/s] 58%| | 253M/436M [00:12<00:09, 18.5MB/s] 58%| | 255M/436M [00:13<00:10, 18.1MB/s] 59%| | 257M/436M [00:13<00:09, 18.4MB/s] 60%| | 260M/436M [00:13<00:09, 19.4MB/s] 60%| | 262M/436M [00:13<00:08, 20.1MB/s] 61%| | 265M/436M [00:13<00:08, 20.6MB/s] 61%| | 267M/436M [00:13<00:08, 20.5MB/s] 62%| | 270M/436M [00:13<00:07, 20.9MB/s] 63%| | 273M/436M [00:13<00:07, 21.1MB/s] 63%| | 275M/436M [00:13<00:07, 21.4MB/s] 64%| | 278M/436M [00:14<00:07, 20.6MB/s] 64%| | 280M/436M [00:14<00:07, 20.4MB/s] 65%| | 282M/436M [00:14<00:08, 18.9MB/s] 65%| | 284M/436M [00:14<00:08, 18.7MB/s] 66%| | 286M/436M [00:14<00:07, 18.7MB/s] 66%| | 288M/436M [00:14<00:08, 18.0MB/s] 67%| | 291M/436M [00:14<00:07, 18.8MB/s] 67%| | 293M/436M [00:14<00:07, 18.6MB/s] 68%| | 295M/436M [00:15<00:07, 19.2MB/s] 68%| | 297M/436M [00:15<00:07, 19.0MB/s] 69%| | 299M/436M [00:15<00:07, 18.7MB/s] 69%| | 301M/436M [00:15<00:09, 14.1MB/s] 70%| | 304M/436M [00:15<00:08, 16.0MB/s] 70%| | 307M/436M [00:15<00:07, 17.5MB/s] 71%| | 309M/436M [00:15<00:06, 18.7MB/s] 72%| | 312M/436M [00:16<00:06, 19.6MB/s] 72%| | 315M/436M [00:16<00:06, 20.2MB/s] 73%| | 317M/436M [00:16<00:05, 20.3MB/s] 73%| | 319M/436M [00:16<00:05, 19.9MB/s] 74%| | 321M/436M [00:16<00:05, 19.7MB/s] 74%| | 323M/436M [00:16<00:05, 19.5MB/s] 75%| | 326M/436M [00:16<00:05, 19.1MB/s] 75%| | 328M/436M [00:16<00:05, 19.1MB/s] 76%| | 330M/436M [00:16<00:05, 19.0MB/s] 76%| | 332M/436M [00:17<00:05, 19.8MB/s] 77%| | 335M/436M [00:17<00:04, 20.5MB/s] 77%| | 337M/436M [00:17<00:04, 20.1MB/s] 78%| | 340M/436M [00:17<00:04, 20.5MB/s] 79%| | 342M/436M [00:17<00:04, 20.8MB/s] 79%| | 345M/436M [00:17<00:04, 21.2MB/s] 80%| | 348M/436M [00:17<00:04, 21.4MB/s] 80%| | 350M/436M [00:17<00:04, 21.3MB/s] 81%| | 353M/436M [00:17<00:03, 21.6MB/s] 82%| | 355M/436M [00:18<00:03, 21.2MB/s] 82%| | 358M/436M [00:18<00:03, 21.9MB/s] 83%| | 361M/436M [00:18<00:03, 21.8MB/s] 83%| | 363M/436M [00:18<00:03, 21.6MB/s] 84%| | 366M/436M [00:18<00:03, 21.0MB/s] 85%| | 369M/436M [00:18<00:03, 19.9MB/s] 85%| | 371M/436M [00:18<00:03, 19.4MB/s] 85%| | 373M/436M [00:18<00:03, 18.8MB/s] 86%| | 375M/436M [00:19<00:03, 18.9MB/s] 86%| | 377M/436M [00:19<00:03, 18.6MB/s] 87%| | 379M/436M [00:19<00:03, 18.3MB/s] 87%| | 381M/436M [00:19<00:02, 18.6MB/s] 88%| | 383M/436M [00:19<00:02, 18.8MB/s] 88%| | 385M/436M [00:19<00:02, 18.9MB/s] 89%| | 387M/436M [00:19<00:02, 19.1MB/s] 89%| | 390M/436M [00:19<00:02, 19.8MB/s] 90%| | 393M/436M [00:20<00:02, 20.5MB/s] 91%| | 395M/436M [00:20<00:01, 20.9MB/s] 91%| | 397M/436M [00:20<00:01, 20.2MB/s] 92%| | 400M/436M [00:20<00:01, 20.7MB/s] 92%| | 402M/436M [00:20<00:01, 19.9MB/s] 93%| | 404M/436M [00:20<00:02, 15.0MB/s] 93%| | 406M/436M [00:20<00:01, 15.7MB/s] 94%| | 409M/436M [00:20<00:01, 17.3MB/s] 94%| | 412M/436M [00:21<00:01, 18.3MB/s] 95%| | 414M/436M [00:21<00:01, 18.6MB/s] 96%| | 417M/436M [00:21<00:00, 19.7MB/s] 96%| | 419M/436M [00:21<00:00, 20.3MB/s] 97%| | 422M/436M [00:21<00:00, 20.7MB/s] 97%| | 425M/436M [00:21<00:00, 21.0MB/s] 98%| | 427M/436M [00:21<00:00, 21.3MB/s] 99%| | 430M/436M [00:21<00:00, 20.5MB/s] 99%| | 433M/436M [00:22<00:00, 20.8MB/s] 100%|| 435M/436M [00:22<00:00, 21.1MB/s] 100%|| 436M/436M [00:22<00:00, 19.6MB/s] ### Make Predictions ```python for (image, mask), _ in zip(val_dataset, range(1)): pred_mask = basnet_model.predict(image) display([image[0], mask[0], normalize_output(pred_mask[0][0])])

1/1 ━━━━━━━━━━━━━━━━━━━━ 0s 32s/step

```  ```
1/1 ━━━━━━━━━━━━━━━━━━━━ 32s 32s/step

png


Relevant Chapters from Deep Learning with Python