I was testing the UNet model along with a mobilenetv2 and saw that there are a lot non-trainable params, which I could not explain. I quick lookup in the model revealed that besides the non-trainable BN params, there are also normal kernels that are marked as non-trainable. I guess this behavior is not desired.
Correct me if I am wrong, but I guess the default trainable parameter should be set to true ?
|
class Upsample_x2_Block(tf.keras.layers.Layer): |
|
""" |
|
""" |
|
def __init__(self, filters, trainable=None): |
|
super(Upsample_x2_Block, self).__init__() |
|
self.trainable = trainable |
Model: "UNet_mobilenetv2_1.00_None"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input (InputLayer) [(None, None, None, 3)] 0
_________________________________________________________________
u_net (UNet) (None, None, None, 30) 15387454
=================================================================
Total params: 15,387,454
Trainable params: 2,384,222
Non-trainable params: 13,003,232
for variable in model.non_trainable_variables:
print(variable.name)
u_net/convolution_bn_activation/batch_normalization/moving_mean:0
u_net/convolution_bn_activation/batch_normalization/moving_variance:0
u_net/convolution_bn_activation_1/batch_normalization_1/moving_mean:0
u_net/convolution_bn_activation_1/batch_normalization_1/moving_variance:0
u_net/upsample_x2__block/conv2d/kernel:0
u_net/upsample_x2__block/conv2d/bias:0
u_net/upsample_x2__block/convolution_bn_activation_2/conv2d_2/kernel:0
u_net/upsample_x2__block/convolution_bn_activation_2/conv2d_2/bias:0
u_net/upsample_x2__block/convolution_bn_activation_2/batch_normalization_2/gamma:0
u_net/upsample_x2__block/convolution_bn_activation_2/batch_normalization_2/beta:0
u_net/upsample_x2__block/convolution_bn_activation_2/batch_normalization_2/moving_mean:0
u_net/upsample_x2__block/convolution_bn_activation_2/batch_normalization_2/moving_variance:0
u_net/upsample_x2__block/convolution_bn_activation_3/conv2d_3/kernel:0
u_net/upsample_x2__block/convolution_bn_activation_3/conv2d_3/bias:0
u_net/upsample_x2__block/convolution_bn_activation_3/batch_normalization_3/gamma:0
u_net/upsample_x2__block/convolution_bn_activation_3/batch_normalization_3/beta:0
u_net/upsample_x2__block/convolution_bn_activation_3/batch_normalization_3/moving_mean:0
u_net/upsample_x2__block/convolution_bn_activation_3/batch_normalization_3/moving_variance:0
u_net/upsample_x2__block_1/conv2d_1/kernel:0
u_net/upsample_x2__block_1/conv2d_1/bias:0
u_net/upsample_x2__block_1/convolution_bn_activation_4/conv2d_4/kernel:0
u_net/upsample_x2__block_1/convolution_bn_activation_4/conv2d_4/bias:0
u_net/upsample_x2__block_1/convolution_bn_activation_4/batch_normalization_4/gamma:0
u_net/upsample_x2__block_1/convolution_bn_activation_4/batch_normalization_4/beta:0
u_net/upsample_x2__block_1/convolution_bn_activation_4/batch_normalization_4/moving_mean:0
u_net/upsample_x2__block_1/convolution_bn_activation_4/batch_normalization_4/moving_variance:0
u_net/upsample_x2__block_1/convolution_bn_activation_5/conv2d_5/kernel:0
u_net/upsample_x2__block_1/convolution_bn_activation_5/conv2d_5/bias:0
u_net/upsample_x2__block_1/convolution_bn_activation_5/batch_normalization_5/gamma:0
u_net/upsample_x2__block_1/convolution_bn_activation_5/batch_normalization_5/beta:0
u_net/upsample_x2__block_1/convolution_bn_activation_5/batch_normalization_5/moving_mean:0
u_net/upsample_x2__block_1/convolution_bn_activation_5/batch_normalization_5/moving_variance:0
u_net/upsample_x2__block_2/conv2d_2/kernel:0
u_net/upsample_x2__block_2/conv2d_2/bias:0
u_net/upsample_x2__block_2/convolution_bn_activation_6/conv2d_6/kernel:0
u_net/upsample_x2__block_2/convolution_bn_activation_6/conv2d_6/bias:0
u_net/upsample_x2__block_2/convolution_bn_activation_6/batch_normalization_6/gamma:0
u_net/upsample_x2__block_2/convolution_bn_activation_6/batch_normalization_6/beta:0
u_net/upsample_x2__block_2/convolution_bn_activation_6/batch_normalization_6/moving_mean:0
u_net/upsample_x2__block_2/convolution_bn_activation_6/batch_normalization_6/moving_variance:0
u_net/upsample_x2__block_2/convolution_bn_activation_7/conv2d_7/kernel:0
u_net/upsample_x2__block_2/convolution_bn_activation_7/conv2d_7/bias:0
u_net/upsample_x2__block_2/convolution_bn_activation_7/batch_normalization_7/gamma:0
u_net/upsample_x2__block_2/convolution_bn_activation_7/batch_normalization_7/beta:0
u_net/upsample_x2__block_2/convolution_bn_activation_7/batch_normalization_7/moving_mean:0
u_net/upsample_x2__block_2/convolution_bn_activation_7/batch_normalization_7/moving_variance:0
u_net/upsample_x2__block_3/conv2d_3/kernel:0
u_net/upsample_x2__block_3/conv2d_3/bias:0
u_net/upsample_x2__block_3/convolution_bn_activation_8/conv2d_8/kernel:0
u_net/upsample_x2__block_3/convolution_bn_activation_8/conv2d_8/bias:0
u_net/upsample_x2__block_3/convolution_bn_activation_8/batch_normalization_8/gamma:0
u_net/upsample_x2__block_3/convolution_bn_activation_8/batch_normalization_8/beta:0
u_net/upsample_x2__block_3/convolution_bn_activation_8/batch_normalization_8/moving_mean:0
u_net/upsample_x2__block_3/convolution_bn_activation_8/batch_normalization_8/moving_variance:0
u_net/upsample_x2__block_3/convolution_bn_activation_9/conv2d_9/kernel:0
u_net/upsample_x2__block_3/convolution_bn_activation_9/conv2d_9/bias:0
u_net/upsample_x2__block_3/convolution_bn_activation_9/batch_normalization_9/gamma:0
u_net/upsample_x2__block_3/convolution_bn_activation_9/batch_normalization_9/beta:0
u_net/upsample_x2__block_3/convolution_bn_activation_9/batch_normalization_9/moving_mean:0
u_net/upsample_x2__block_3/convolution_bn_activation_9/batch_normalization_9/moving_variance:0
u_net/upsample_x2__block_4/conv2d_4/kernel:0
u_net/upsample_x2__block_4/conv2d_4/bias:0
u_net/upsample_x2__block_4/convolution_bn_activation_10/conv2d_10/kernel:0
u_net/upsample_x2__block_4/convolution_bn_activation_10/conv2d_10/bias:0
u_net/upsample_x2__block_4/convolution_bn_activation_10/batch_normalization_10/gamma:0
u_net/upsample_x2__block_4/convolution_bn_activation_10/batch_normalization_10/beta:0
u_net/upsample_x2__block_4/convolution_bn_activation_10/batch_normalization_10/moving_mean:0
u_net/upsample_x2__block_4/convolution_bn_activation_10/batch_normalization_10/moving_variance:0
u_net/upsample_x2__block_4/convolution_bn_activation_11/conv2d_11/kernel:0
u_net/upsample_x2__block_4/convolution_bn_activation_11/conv2d_11/bias:0
u_net/upsample_x2__block_4/convolution_bn_activation_11/batch_normalization_11/gamma:0
u_net/upsample_x2__block_4/convolution_bn_activation_11/batch_normalization_11/beta:0
u_net/upsample_x2__block_4/convolution_bn_activation_11/batch_normalization_11/moving_mean:0
u_net/upsample_x2__block_4/convolution_bn_activation_11/batch_normalization_11/moving_variance:0
I was testing the UNet model along with a mobilenetv2 and saw that there are a lot non-trainable params, which I could not explain. I quick lookup in the model revealed that besides the non-trainable BN params, there are also normal kernels that are marked as non-trainable. I guess this behavior is not desired.
Correct me if I am wrong, but I guess the default
trainableparameter should be set totrue?TensorFlow-Advanced-Segmentation-Models/tensorflow_advanced_segmentation_models/models/_custom_layers_and_blocks.py
Lines 214 to 219 in c06d3d2