做网站和做系统有什么区别,网站色差表,合肥知名建站公司,软件外包服务是什么意思ResNet50卷积神经网络输出数据形参分析-笔记
ResNet50包含多个模块#xff0c;其中第2到第5个模块分别包含3、4、6、3个残差块 5049个卷积#xff08;3463)*31和一个全连接层 分析结果为#xff1a; 输入数据形状:[10, 3, 224, 224] 最后输出结果#xff1a;linear_0 [10,…ResNet50卷积神经网络输出数据形参分析-笔记
ResNet50包含多个模块其中第2到第5个模块分别包含3、4、6、3个残差块 5049个卷积3463)*31和一个全连接层 分析结果为 输入数据形状:[10, 3, 224, 224] 最后输出结果linear_0 [10, 1] [2048, 1] [1] ResNet50包含多个模块其中第2到第5个模块分别包含3、4、6、3个残差块 第1模块7X7(64) 一个卷积 第2模块3X3(64) 三个残差块9个卷积 第3模块3X3(128) 四个残差块12个卷积 第4模块3X3(256) 六个残差块18个卷积 第5模块3X3(512) 三个残差块9个卷积 最后一个全连接层
分析详细过程如下所示
PS E:\project\python D:/ProgramData/Anaconda3/python.exe e:/project/python/PM/ResNet_PM_test.py
layers 50
W0804 20:41:04.044713 18388 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 6.1, Driver API Version: 12.2, Runtime API Version: 10.2
W0804 20:41:04.053730 18388 gpu_resources.cc:91] device: 0, cuDNN Version: 7.6.
block 0 depth[block] 3 Cout256
bottleneck[ 256 64 False
bottleneck[ 256 64 True
bottleneck[ 256 64 True
create bnb 3
block 1 depth[block] 4 Cout512
bottleneck[ 512 128 False
bottleneck[ 512 128 True
bottleneck[ 512 128 True
bottleneck[ 512 128 True
create bnb 4
block 2 depth[block] 6 Cout1024
bottleneck[ 1024 256 False
bottleneck[ 1024 256 True
bottleneck[ 1024 256 True
bottleneck[ 1024 256 True
bottleneck[ 1024 256 True
bottleneck[ 1024 256 True
create bnb 6
block 3 depth[block] 3 Cout2048
bottleneck[ 2048 512 False
bottleneck[ 2048 512 True
bottleneck[ 2048 512 True
create bnb 3
[10, 3, 224, 224]
conv2d_0 [10, 64, 112, 112] [64, 3, 7, 7] [ 64 ]
D:\ProgramData\Anaconda3\lib\site-packages\paddle\nn\layer\norm.py:712: UserWarning: When training, we now always track global mean and variance.warnings.warn(
conv_bn_layer_0 [10, 64, 112, 112]
maxpool2d: [10, 64, 56, 56]
start bottleneckBlock:
#第二个模块 Cout256 包括三个残差块bottleneckBlock
-----print bottleneckBlock: 0
conv2d_1 [10, 64, 56, 56] [64, 64, 1, 1] [ 64 ]
conv2d_2 [10, 64, 56, 56] [64, 64, 3, 3] [ 64 ]
conv2d_3 [10, 256, 56, 56] [256, 64, 1, 1] [ 256 ]
conv2d_4 [10, 256, 56, 56] [256, 64, 1, 1] [ 256 ]
end block: [10, 256, 56, 56]
-----print bottleneckBlock: 1
conv2d_5 [10, 64, 56, 56] [64, 256, 1, 1] [ 64 ]
conv2d_6 [10, 64, 56, 56] [64, 64, 3, 3] [ 64 ]
conv2d_7 [10, 256, 56, 56] [256, 64, 1, 1] [ 256 ]
end block: [10, 256, 56, 56]
-----print bottleneckBlock: 2
conv2d_8 [10, 64, 56, 56] [64, 256, 1, 1] [ 64 ]
conv2d_9 [10, 64, 56, 56] [64, 64, 3, 3] [ 64 ]
conv2d_10 [10, 256, 56, 56] [256, 64, 1, 1] [ 256 ]
end block: [10, 256, 56, 56]
#第三个模块 Cout512 包括四个残差块bottleneckBlock
-----print bottleneckBlock: 3
conv2d_11 [10, 128, 56, 56] [128, 256, 1, 1] [ 128 ]
conv2d_12 [10, 128, 28, 28] [128, 128, 3, 3] [ 128 ]
conv2d_13 [10, 512, 28, 28] [512, 128, 1, 1] [ 512 ]
conv2d_14 [10, 512, 28, 28] [512, 256, 1, 1] [ 512 ]
end block: [10, 512, 28, 28]
-----print bottleneckBlock: 4
conv2d_15 [10, 128, 28, 28] [128, 512, 1, 1] [ 128 ]
conv2d_16 [10, 128, 28, 28] [128, 128, 3, 3] [ 128 ]
conv2d_17 [10, 512, 28, 28] [512, 128, 1, 1] [ 512 ]
end block: [10, 512, 28, 28]
-----print bottleneckBlock: 5
conv2d_18 [10, 128, 28, 28] [128, 512, 1, 1] [ 128 ]
conv2d_19 [10, 128, 28, 28] [128, 128, 3, 3] [ 128 ]
conv2d_20 [10, 512, 28, 28] [512, 128, 1, 1] [ 512 ]
end block: [10, 512, 28, 28]
-----print bottleneckBlock: 6
conv2d_21 [10, 128, 28, 28] [128, 512, 1, 1] [ 128 ]
conv2d_22 [10, 128, 28, 28] [128, 128, 3, 3] [ 128 ]
conv2d_23 [10, 512, 28, 28] [512, 128, 1, 1] [ 512 ]
end block: [10, 512, 28, 28]
#第四个模块 Cout1024 包括六个残差块bottleneckBlock
-----print bottleneckBlock: 7
conv2d_24 [10, 256, 28, 28] [256, 512, 1, 1] [ 256 ]
conv2d_25 [10, 256, 14, 14] [256, 256, 3, 3] [ 256 ]
conv2d_26 [10, 1024, 14, 14] [1024, 256, 1, 1] [ 1024 ]
conv2d_27 [10, 1024, 14, 14] [1024, 512, 1, 1] [ 1024 ]
end block: [10, 1024, 14, 14]
-----print bottleneckBlock: 8
conv2d_28 [10, 256, 14, 14] [256, 1024, 1, 1] [ 256 ]
conv2d_29 [10, 256, 14, 14] [256, 256, 3, 3] [ 256 ]
conv2d_30 [10, 1024, 14, 14] [1024, 256, 1, 1] [ 1024 ]
end block: [10, 1024, 14, 14]
-----print bottleneckBlock: 9
conv2d_31 [10, 256, 14, 14] [256, 1024, 1, 1] [ 256 ]
conv2d_32 [10, 256, 14, 14] [256, 256, 3, 3] [ 256 ]
conv2d_33 [10, 1024, 14, 14] [1024, 256, 1, 1] [ 1024 ]
end block: [10, 1024, 14, 14]
-----print bottleneckBlock: 10
conv2d_34 [10, 256, 14, 14] [256, 1024, 1, 1] [ 256 ]
conv2d_35 [10, 256, 14, 14] [256, 256, 3, 3] [ 256 ]
conv2d_36 [10, 1024, 14, 14] [1024, 256, 1, 1] [ 1024 ]
end block: [10, 1024, 14, 14]
-----print bottleneckBlock: 11
conv2d_37 [10, 256, 14, 14] [256, 1024, 1, 1] [ 256 ]
conv2d_38 [10, 256, 14, 14] [256, 256, 3, 3] [ 256 ]
conv2d_39 [10, 1024, 14, 14] [1024, 256, 1, 1] [ 1024 ]
end block: [10, 1024, 14, 14]
-----print bottleneckBlock: 12
conv2d_40 [10, 256, 14, 14] [256, 1024, 1, 1] [ 256 ]
conv2d_41 [10, 256, 14, 14] [256, 256, 3, 3] [ 256 ]
conv2d_42 [10, 1024, 14, 14] [1024, 256, 1, 1] [ 1024 ]
end block: [10, 1024, 14, 14]
#第五个模块 Cout2048 包括三个残差块bottleneckBlock
-----print bottleneckBlock: 13
conv2d_43 [10, 512, 14, 14] [512, 1024, 1, 1] [ 512 ]
conv2d_44 [10, 512, 7, 7] [512, 512, 3, 3] [ 512 ]
conv2d_45 [10, 2048, 7, 7] [2048, 512, 1, 1] [ 2048 ]
conv2d_46 [10, 2048, 7, 7] [2048, 1024, 1, 1] [ 2048 ]
end block: [10, 2048, 7, 7]
-----print bottleneckBlock: 14
conv2d_47 [10, 512, 7, 7] [512, 2048, 1, 1] [ 512 ]
conv2d_48 [10, 512, 7, 7] [512, 512, 3, 3] [ 512 ]
conv2d_49 [10, 2048, 7, 7] [2048, 512, 1, 1] [ 2048 ]
end block: [10, 2048, 7, 7]
-----print bottleneckBlock: 15
conv2d_50 [10, 512, 7, 7] [512, 2048, 1, 1] [ 512 ]
conv2d_51 [10, 512, 7, 7] [512, 512, 3, 3] [ 512 ]
conv2d_52 [10, 2048, 7, 7] [2048, 512, 1, 1] [ 2048 ]
end block: [10, 2048, 7, 7]
end bottleneckBlock:
adaptive_avg_pool2d_0 [10, 2048, 1, 1]
y.shape [10, 2048]
linear_0 [10, 1] [2048, 1] [1]
PS E:\project\python 分析测试代码如下所示
# -*- coding:utf-8 -*-# ResNet模型代码
import numpy as np
import paddle
import paddle.nn as nn
import paddle.nn.functional as Fdef printItem(item,x):# item是CNN类中的一个子层# 查看经过子层之后的输出数据形状try:x item(x)except:x paddle.reshape(x, [x.shape[0], -1])x item(x) #print(len(item.parameters())) if len(item.parameters())1: print(item.full_name(), x.shape,item.parameters()[0].shape,[,item.parameters()[0].shape[0],]) #print(item)elif len(item.parameters())2:# 查看卷积和全连接层的数据和参数的形状# 其中item.parameters()[0]是权重参数witem.parameters()[1]是偏置参数bprint(item.full_name(), x.shape, item.parameters()[0].shape, item.parameters()[1].shape)else:# 池化层没有参数print(item.full_name(), x.shape) return x; # ResNet中使用了BatchNorm层在卷积层的后面加上BatchNorm以提升数值稳定性
# 定义卷积批归一化块 (包括一个卷积)
class ConvBNLayer(paddle.nn.Layer):def __init__(self,num_channels,num_filters,filter_size,stride1,groups1,actNone):num_channels, 卷积层的输入通道数num_filters, 卷积层的输出通道数stride, 卷积层的步幅groups, 分组卷积的组数默认groups1不使用分组卷积super(ConvBNLayer, self).__init__()# 创建卷积层self._conv nn.Conv2D(in_channelsnum_channels,out_channelsnum_filters,kernel_sizefilter_size,stridestride,padding(filter_size - 1) // 2,groupsgroups,bias_attrFalse)# 创建BatchNorm层self._batch_norm paddle.nn.BatchNorm2D(num_filters)self.act actdef forward_old(self, inputs):y self._conv(inputs)y self._batch_norm(y)if self.act leaky:y F.leaky_relu(xy, negative_slope0.1)elif self.act relu:y F.relu(xy)return ydef forward(self, inputs):yprintItem(self._conv,inputs)#print([,num_filters,num_channels,filter_size,filter_size)y self._batch_norm(y)if self.act leaky:y F.leaky_relu(xy, negative_slope0.1)elif self.act relu:y F.relu(xy)return y# 定义残差块 (包括三个卷积) 16*348
# 每个残差块会对输入图片做三次卷积然后跟输入图片进行短接
# 如果残差块中第三次卷积输出特征图的形状与输入不一致则对输入图片做1x1卷积将其输出形状调整成一致
class BottleneckBlock(paddle.nn.Layer):def __init__(self,num_channels,num_filters,stride,shortcutTrue):super(BottleneckBlock, self).__init__()# 创建第一个卷积层 1x1self.conv0 ConvBNLayer(num_channelsnum_channels,num_filtersnum_filters,filter_size1,actrelu)# 创建第二个卷积层 3x3self.conv1 ConvBNLayer(num_channelsnum_filters,num_filtersnum_filters,filter_size3,stridestride,actrelu)# 创建第三个卷积 1x1但输出通道数乘以4self.conv2 ConvBNLayer(num_channelsnum_filters,num_filtersnum_filters * 4,filter_size1,actNone)# 如果conv2的输出跟此残差块的输入数据形状一致则shortcutTrue# 否则shortcut False添加1个1x1的卷积作用在输入数据上使其形状变成跟conv2一致if not shortcut:self.short ConvBNLayer(num_channelsnum_channels,num_filtersnum_filters * 4,filter_size1,stridestride)self.shortcut shortcutself._num_channels_out num_filters * 4def forward_old(self, inputs):y self.conv0(inputs)conv1 self.conv1(y)conv2 self.conv2(conv1)# 如果shortcutTrue直接将inputs跟conv2的输出相加# 否则需要对inputs进行一次卷积将形状调整成跟conv2输出一致if self.shortcut:short inputselse:short self.short(inputs)y paddle.add(xshort, yconv2)y F.relu(y)return ydef forward(self, inputs): y self.conv0(inputs)#print(ConvBMLayer0.shape,y.shape,self.conv0)conv1 self.conv1(y)#print(ConvBMLayer1.shape,conv1.shape)conv2 self.conv2(conv1)#print(ConvBMLayer2.shape,conv2.shape)# 如果shortcutTrue直接将inputs跟conv2的输出相加# 否则需要对inputs进行一次卷积将形状调整成跟conv2输出一致if self.shortcut:short inputselse:short self.short(inputs)y paddle.add(xshort, yconv2)y F.relu(y)return y # 定义ResNet模型
class ResNet(paddle.nn.Layer):def __init__(self, layers50, class_dim1):print(layers,layers)layers, 网络层数可以是50, 101或者152class_dim分类标签的类别数super(ResNet, self).__init__()self.layers layerssupported_layers [50, 101, 152]assert layers in supported_layers, \supported layers are {} but input layer is {}.format(supported_layers, layers)if layers 50:#ResNet50包含多个模块其中第2到第5个模块分别包含3、4、6、3个残差块depth [3, 4, 6, 3]elif layers 101:#ResNet101包含多个模块其中第2到第5个模块分别包含3、4、23、3个残差块depth [3, 4, 23, 3]elif layers 152:#ResNet152包含多个模块其中第2到第5个模块分别包含3、8、36、3个残差块depth [3, 8, 36, 3]# 残差块中使用到的卷积的输出通道数num_filters [64, 128, 256, 512]# ResNet的第一个模块包含1个7x7卷积后面跟着1个最大池化层self.conv ConvBNLayer(num_channels3,num_filters64,filter_size7,stride2,actrelu)self.pool2d_max nn.MaxPool2D(kernel_size3,stride2,padding1)# ResNet的第二到第五个模块c2、c3、c4、c5self.bottleneck_block_list []num_channels 64for block in range(len(depth)): #4(0,1,2,3)shortcut Falseprint(block,block,depth[block],depth[block])k0for i in range(depth[block]): #depth [3, 4, 6, 3]k1# c3、c4、c5将会在第一个残差块使用stride2其余所有残差块stride1bottleneck_block self.add_sublayer(bb_%d_%d % (block, i),BottleneckBlock(num_channelsnum_channels,num_filtersnum_filters[block],stride2 if i 0 and block ! 0 else 1, shortcutshortcut))num_channels bottleneck_block._num_channels_outself.bottleneck_block_list.append(bottleneck_block)print(bottleneck[,num_channels,num_filters[block],shortcut)shortcut True; print(create bnb,k)# 在c5的输出特征图上使用全局池化self.pool2d_avg paddle.nn.AdaptiveAvgPool2D(output_size1)# stdv用来作为全连接层随机初始化参数的方差import mathstdv 1.0 / math.sqrt(2048 * 1.0)# 创建全连接层输出大小为类别数目经过残差网络的卷积和全局池化后# 卷积特征的维度是[B,2048,1,1]故最后一层全连接的输入维度是2048self.out nn.Linear(in_features2048, out_featuresclass_dim,weight_attrpaddle.ParamAttr(initializerpaddle.nn.initializer.Uniform(-stdv, stdv)))def forward(self, inputs):y self.conv(inputs)y self.pool2d_max(y)for bottleneck_block in self.bottleneck_block_list:y bottleneck_block(y)y self.pool2d_avg(y)y paddle.reshape(y, [y.shape[0], -1])y self.out(y)return ydef printStruct(self,inputs):ypaddle.to_tensor(inputs)print(y.shape)yprintItem(self.conv,y)y self.pool2d_max(y)print(maxpool2d:,y.shape)print(start bottleneckBlock:)i0 for bottleneck_block in self.bottleneck_block_list: print(-----print bottleneckBlock:,i)y bottleneck_block(y)print(end block:,y.shape)i1 print(end bottleneckBlock:)yprintItem(self.pool2d_avg,y)y paddle.reshape(y, [y.shape[0], -1])print(y.shape,y.shape) yprintItem(self.out,y)return y# 创建模型
model ResNet()
# 定义优化器
opt paddle.optimizer.Momentum(learning_rate0.001, momentum0.9, parametersmodel.parameters(), weight_decay0.001)
# 启动训练过程
import PM
#PM.train_pm(model, opt)
## 输入数据形状是 [N, 3, H, W]
# 这里用np.random创建一个随机数组作为输入数据
x np.random.randn(*[10,3,224,224])
x x.astype(float32)
# 创建CNN类的实例指定模型名称和分类的类别数目
#model VGG(1)
#
model.printStruct(x)
#训练源代码如下所示
# -*- coding:utf-8 -*-# ResNet模型代码
import numpy as np
import paddle
import paddle.nn as nn
import paddle.nn.functional as F# ResNet中使用了BatchNorm层在卷积层的后面加上BatchNorm以提升数值稳定性
# 定义卷积批归一化块 (包括一个卷积)
class ConvBNLayer(paddle.nn.Layer):def __init__(self,num_channels,num_filters,filter_size,stride1,groups1,actNone):num_channels, 卷积层的输入通道数num_filters, 卷积层的输出通道数stride, 卷积层的步幅groups, 分组卷积的组数默认groups1不使用分组卷积super(ConvBNLayer, self).__init__()# 创建卷积层self._conv nn.Conv2D(in_channelsnum_channels,out_channelsnum_filters,kernel_sizefilter_size,stridestride,padding(filter_size - 1) // 2,groupsgroups,bias_attrFalse)# 创建BatchNorm层self._batch_norm paddle.nn.BatchNorm2D(num_filters)self.act actdef forward(self, inputs):y self._conv(inputs)y self._batch_norm(y)if self.act leaky:y F.leaky_relu(xy, negative_slope0.1)elif self.act relu:y F.relu(xy)return y# 定义残差块 (包括三个卷积)
# 每个残差块会对输入图片做三次卷积然后跟输入图片进行短接
# 如果残差块中第三次卷积输出特征图的形状与输入不一致则对输入图片做1x1卷积将其输出形状调整成一致
class BottleneckBlock(paddle.nn.Layer):def __init__(self,num_channels,num_filters,stride,shortcutTrue):super(BottleneckBlock, self).__init__()# 创建第一个卷积层 1x1self.conv0 ConvBNLayer(num_channelsnum_channels,num_filtersnum_filters,filter_size1,actrelu)# 创建第二个卷积层 3x3self.conv1 ConvBNLayer(num_channelsnum_filters,num_filtersnum_filters,filter_size3,stridestride,actrelu)# 创建第三个卷积 1x1但输出通道数乘以4self.conv2 ConvBNLayer(num_channelsnum_filters,num_filtersnum_filters * 4,filter_size1,actNone)# 如果conv2的输出跟此残差块的输入数据形状一致则shortcutTrue# 否则shortcut False添加1个1x1的卷积作用在输入数据上使其形状变成跟conv2一致if not shortcut:self.short ConvBNLayer(num_channelsnum_channels,num_filtersnum_filters * 4,filter_size1,stridestride)self.shortcut shortcutself._num_channels_out num_filters * 4def forward(self, inputs):y self.conv0(inputs)conv1 self.conv1(y)conv2 self.conv2(conv1)# 如果shortcutTrue直接将inputs跟conv2的输出相加# 否则需要对inputs进行一次卷积将形状调整成跟conv2输出一致if self.shortcut:short inputselse:short self.short(inputs)y paddle.add(xshort, yconv2)y F.relu(y)return y# 定义ResNet模型
class ResNet(paddle.nn.Layer):def __init__(self, layers50, class_dim1):layers, 网络层数可以是50, 101或者152class_dim分类标签的类别数super(ResNet, self).__init__()self.layers layerssupported_layers [50, 101, 152]assert layers in supported_layers, \supported layers are {} but input layer is {}.format(supported_layers, layers)if layers 50:#ResNet50包含多个模块其中第2到第5个模块分别包含3、4、6、3个残差块depth [3, 4, 6, 3]elif layers 101:#ResNet101包含多个模块其中第2到第5个模块分别包含3、4、23、3个残差块depth [3, 4, 23, 3]elif layers 152:#ResNet152包含多个模块其中第2到第5个模块分别包含3、8、36、3个残差块depth [3, 8, 36, 3]# 残差块中使用到的卷积的输出通道数num_filters [64, 128, 256, 512]# ResNet的第一个模块包含1个7x7卷积后面跟着1个最大池化层self.conv ConvBNLayer(num_channels3,num_filters64,filter_size7,stride2,actrelu)self.pool2d_max nn.MaxPool2D(kernel_size3,stride2,padding1)# ResNet的第二到第五个模块c2、c3、c4、c5self.bottleneck_block_list []num_channels 64for block in range(len(depth)):shortcut Falsefor i in range(depth[block]):# c3、c4、c5将会在第一个残差块使用stride2其余所有残差块stride1bottleneck_block self.add_sublayer(bb_%d_%d % (block, i),BottleneckBlock(num_channelsnum_channels,num_filtersnum_filters[block],stride2 if i 0 and block ! 0 else 1, shortcutshortcut))num_channels bottleneck_block._num_channels_outself.bottleneck_block_list.append(bottleneck_block)shortcut True# 在c5的输出特征图上使用全局池化self.pool2d_avg paddle.nn.AdaptiveAvgPool2D(output_size1)# stdv用来作为全连接层随机初始化参数的方差import mathstdv 1.0 / math.sqrt(2048 * 1.0)# 创建全连接层输出大小为类别数目经过残差网络的卷积和全局池化后# 卷积特征的维度是[B,2048,1,1]故最后一层全连接的输入维度是2048self.out nn.Linear(in_features2048, out_featuresclass_dim,weight_attrpaddle.ParamAttr(initializerpaddle.nn.initializer.Uniform(-stdv, stdv)))def forward(self, inputs):y self.conv(inputs)y self.pool2d_max(y)for bottleneck_block in self.bottleneck_block_list:y bottleneck_block(y)y self.pool2d_avg(y)y paddle.reshape(y, [y.shape[0], -1])y self.out(y)return y
# 创建模型
model ResNet() #ResNet(50) ResNet(101) ResNet(152)
# 定义优化器
opt paddle.optimizer.Momentum(learning_rate0.001, momentum0.9, parametersmodel.parameters(), weight_decay0.001)
# 启动训练过程
import PM
PM.train_pm(model, opt)
#训练结果
D:\ProgramData\Anaconda3\lib\site-packages\paddle\nn\layer\norm.py:712: UserWarning: When training, we now always track global mean and variance.warnings.warn(
epoch: 0, batch_id: 0, loss is: 0.7711
epoch: 0, batch_id: 20, loss is: 0.6860
[validation] accuracy/loss: 0.7700/0.4910
epoch: 1, batch_id: 0, loss is: 0.7769
epoch: 1, batch_id: 20, loss is: 0.6261
[validation] accuracy/loss: 0.8475/0.3368
epoch: 2, batch_id: 0, loss is: 0.4543
epoch: 2, batch_id: 20, loss is: 0.3392
[validation] accuracy/loss: 0.8950/0.2690
epoch: 3, batch_id: 0, loss is: 1.1716
epoch: 3, batch_id: 20, loss is: 0.0736
[validation] accuracy/loss: 0.8975/0.2387
epoch: 4, batch_id: 0, loss is: 0.0909
epoch: 4, batch_id: 20, loss is: 0.1900
[validation] accuracy/loss: 0.9375/0.2098
PS E:\project\python
