目录

🌊1. 研究目的

🌊2. 研究准备

🌊3. 研究内容

🌍3.1 多层感知机模型选择、欠拟合和过拟合

🌍3.2 练习

🌊4. 研究体会

🌊1. 研究目的

理解块的网络结构；比较块的网络与传统浅层网络的性能差异；探究块的网络深度与性能之间的关系；研究块的网络在不同任务上的适用性。

🌊2. 研究准备

根据GPU安装pytorch版本实现GPU运行研究代码；配置环境用来运行 Python、Jupyter Notebook和相关库等相关库。

🌊3. 研究内容

启动jupyter notebook，使用新增的pytorch环境新建ipynb文件，为了检查环境配置是否合理，输入import torch以及torch.cuda.is_available() ，若返回TRUE则说明研究环境配置正确，若返回False但可以正确导入torch则说明pytorch配置成功，但研究运行是在CPU进行的，结果如下：

🌍3.1 使用块的网络（VGG）

（1）使用jupyter notebook新增的pytorch环境新建ipynb文件，完成基本数据操作的研究代码与练习结果如下：

VGG块

<code>import torch

from torch import nn

from d2l import torch as d2l

def vgg_block(num_convs, in_channels, out_channels):

layers = []

for _ in range(num_convs):

layers.append(nn.Conv2d(in_channels, out_channels,

kernel_size=3, padding=1))

layers.append(nn.ReLU())

in_channels = out_channels

layers.append(nn.MaxPool2d(kernel_size=2,stride=2))

return nn.Sequential(*layers)

VGG网络

conv_arch = ((1, 64), (1, 128), (2, 256), (2, 512), (2, 512))

def vgg(conv_arch):

conv_blks = []

in_channels = 1

# 卷积层部分

for (num_convs, out_channels) in conv_arch:

conv_blks.append(vgg_block(num_convs, in_channels, out_channels))

in_channels = out_channels

return nn.Sequential(

*conv_blks, nn.Flatten(),

# 全连接层部分

nn.Linear(out_channels * 7 * 7, 4096), nn.ReLU(), nn.Dropout(0.5),

nn.Linear(4096, 4096), nn.ReLU(), nn.Dropout(0.5),

nn.Linear(4096, 10))

net = vgg(conv_arch)

X = torch.randn(size=(1, 1, 224, 224))

for blk in net:

X = blk(X)

print(blk.__class__.__name__,'output shape:\t',X.shape)

训练模型

<code>ratio = 4

small_conv_arch = [(pair[0], pair[1] // ratio) for pair in conv_arch]

net = vgg(small_conv_arch)

lr, num_epochs, batch_size = 0.05, 10, 128

train_iter, test_iter = d2l.load_data_fashion_mnist(batch_size, resize=224)

d2l.train_ch6(net, train_iter, test_iter, num_epochs, lr, d2l.try_gpu())

🌍3.2 练习

1.打印层的尺寸时，我们只看到8个结果，而不是11个结果。剩余的3层信息去哪了？

在VGG网络中，最后三个全连接层（nn.Linear）的输出尺寸没有被打印出来。这是因为在for循环中，只遍历了卷积层部分的模块，而没有遍历全连接层部分的模块。

为了打印出全连接层的输出尺寸，可以将全连接层的模块添加到一个列表中，然后在遍历网络的时候也打印出这些模块的输出尺寸。以下是修改后的代码：

<code>import torch

import torch.nn as nn

# 定义 VGG 模型

def vgg(conv_arch):

layers = []

in_channels = 1

for c, num_convs in conv_arch:

layers.append(nn.Sequential(

nn.Conv2d(in_channels, c, kernel_size=3, padding=1),

nn.ReLU()

))

in_channels = c

for _ in range(num_convs - 1):

layers.append(nn.Sequential(

nn.Conv2d(c, c, kernel_size=3, padding=1),

nn.ReLU()

))

layers.append(nn.MaxPool2d(kernel_size=2, stride=2))

return nn.Sequential(*layers)

conv_arch = ((1, 1), (2, 2), (4, 2), (8, 2), (16, 2))

net = vgg(conv_arch)

X = torch.randn(size=(1, 1, 224, 224))

for blk in net:

X = blk(X)

print(blk.__class__.__name__, 'output shape:\t', X.shape)

# 获取卷积层的输出尺寸

_, channels, height, width = X.shape

# 添加全连接层模块到列表中

fc_layers = [

nn.Flatten(),

nn.Linear(channels * height * width, 4096),

nn.ReLU(),

nn.Dropout(0.5),

nn.Linear(4096, 4096),

nn.ReLU(),

nn.Dropout(0.5),

nn.Linear(4096, 10)

]

net = nn.Sequential(*net, *fc_layers)

# 打印全连接层的输出尺寸

for blk in fc_layers:

X = blk(X)

print(blk.__class__.__name__, 'output shape:\t', X.shape)

2.与AlexNet相比，VGG的计算要慢得多，而且它还需要更多的显存。分析出现这种情况的原因。

VGG相对于AlexNet而言，计算速度更慢、显存需求更高的主要原因如下：

更深的网络结构： VGG网络相比AlexNet更深，它使用了更多的卷积层和全连接层。VGG的网络结构包含了多个连续的卷积层，导致了参数的数量增加和计算量的增加。相对于AlexNet的8层网络，VGG网络有16或19层，这导致了更多的计算和内存消耗。小尺寸的卷积核： VGG网络使用了较小的卷积核（3x3大小），而AlexNet则使用了更大的卷积核（11x11和5x5大小）。较小的卷积核意味着每个卷积层需要进行更多次的卷积运算来覆盖相同的感受野，从而增加了计算量。更多的参数： VGG网络具有更多的参数量。由于每个卷积层都使用了较小的卷积核，导致了更多的卷积核参数。此外，由于网络更深，全连接层的参数数量也更多。更多的参数需要更多的内存来存储，并且在计算过程中需要更多的计算量。更高的内存需求： VGG网络的深度和参数量的增加导致了更高的内存需求。在训练过程中，需要存储每一层的输出、梯度和参数，这会消耗大量的显存。较大的显存需求可能导致无法一次性加载更多的数据和模型，从而导致更多的显存读写操作和内存碎片化，进而影响计算速度。

3.尝试将Fashion-MNIST数据集图像的高度和宽度从224改为96。这对实验有什么影响？

将Fashion-MNIST数据集图像的高度和宽度从224改为96会对实验产生以下影响：

模型性能下降： 缩小图像尺寸会导致图像的信息丢失和细节损失。VGG网络在设计时使用了较大的输入尺寸，以便更好地捕捉图像的细节和纹理。通过将图像尺寸缩小为96x96，模型可能无法有效地捕捉到原始图像中的细微特征，从而导致性能下降。空间分辨率减小： 图像尺寸从224x224缩小到96x96会导致空间分辨率的减小。较小的图像尺寸意味着每个像素代表的区域更大，图像中的细节信息被压缩。这可能会导致模型在进行物体边界检测、纹理识别等任务时表现不佳。减少计算和内存需求： 缩小图像尺寸可以减少模型的计算和内存需求。较小的输入图像尺寸意味着每个卷积层的特征图大小也会减小，从而减少了参数数量和计算量。此外，由于特征图大小减小，模型所需的内存也会相应减少。训练速度加快： 缩小图像尺寸可以加快训练速度。较小的输入图像尺寸意味着每个批次的数据量减少，从而减少了每个训练步骤的计算量和内存消耗。这可能导致更快的训练速度和更高的训练效率。

<code>import torch

from torch import nn

from torchvision.transforms import ToTensor

from torchvision.datasets import FashionMNIST

from torch.utils.data import DataLoader

# 定义VGG块

def vgg_block(num_convs, in_channels, out_channels):

layers = []

for _ in range(num_convs):

layers.append(nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1))

layers.append(nn.ReLU())

in_channels = out_channels

layers.append(nn.MaxPool2d(kernel_size=2, stride=2))

return nn.Sequential(*layers)

# 定义VGG网络

class VGG(nn.Module):

def __init__(self, conv_arch):

super(VGG, self).__init__()

self.conv_blocks = nn.Sequential()

in_channels = 1

for i, (num_convs, out_channels) in enumerate(conv_arch):

self.conv_blocks.add_module(f'vgg_block{i+1}', vgg_block(num_convs, in_channels, out_channels))

in_channels = out_channels

self.fc_layers = nn.Sequential(

nn.Linear(out_channels * 3 * 3, 4096),

nn.ReLU(),

nn.Dropout(0.5),

nn.Linear(4096, 4096),

nn.ReLU(),

nn.Dropout(0.5),

nn.Linear(4096, 10)

)

def forward(self, x):

x = self.conv_blocks(x)

x = torch.flatten(x, start_dim=1)

x = self.fc_layers(x)

return x

# 将Fashion-MNIST图像尺寸缩小为96x96

transform = ToTensor()

train_dataset = FashionMNIST(root='./data', train=True, download=True, transform=transform)code>

test_dataset = FashionMNIST(root='./data', train=False, download=True, transform=transform)code>

# 创建数据加载器

batch_size = 128

train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

test_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False)

# 定义VGG网络结构

conv_arch = [(1, 64), (1, 128), (2, 256), (2, 512), (2, 512)]

model = VGG(conv_arch)

# 训练和测试

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

model.to(device)

optimizer = torch.optim.SGD(model.parameters(), lr=0.05)

criterion = nn.CrossEntropyLoss()

def train(model, train_loader, optimizer, criterion, device):

model.train()

train_loss = 0.0

train_acc = 0.0

for images, labels in train_loader:

images = images.to(device)

labels = labels.to(device)

optimizer.zero_grad()

outputs = model(images)

loss = criterion(outputs, labels)

loss.backward()

optimizer.step()

train_loss += loss.item() * images.size(0)

_, preds = torch.max(outputs, 1)

train_acc += torch.sum(preds == labels.data).item()

train_loss /= len(train_loader.dataset)

train_acc /= len(train_loader.dataset)

return train_loss, train_acc

def test(model, test_loader, criterion, device):

model.eval()

test_loss = 0.0

test_acc = 0.0

with torch.no_grad():

for images, labels in test_loader:

images = images.to(device)

labels = labels.to(device)

outputs = model(images)

loss = criterion(outputs, labels)

test_loss += loss.item() * images.size(0)

_, preds = torch.max(outputs, 1)

test_acc += torch.sum(preds == labels.data).item()

test_loss /= len(test_loader.dataset)

test_acc /= len(test_loader.dataset)

return test_loss, test_acc

# 训练模型

num_epochs = 10

for epoch in range(num_epochs):

train_loss, train_acc = train(model, train_loader, optimizer, criterion, device)

test_loss, test_acc = test(model, test_loader, criterion, device)

print(f'Epoch {epoch+1}/{num_epochs}, Train Loss: {train_loss:.4f}, Train Accuracy: {train_acc:.4f}, '

f'Test Loss: {test_loss:.4f}, Test Accuracy: {test_acc:.4f}')

# 输出样例

print("Sample outputs:")

model.eval()

with torch.no_grad():

for images, labels in test_loader:

images = images.to(device)

labels = labels.to(device)

outputs = model(images)

_, preds = torch.max(outputs, 1)

for i in range(images.size(0)):

print(f"Predicted: {preds[i].item()}, Ground Truth: {labels[i].item()}")

break # 只输出一个批次的样例结果

在上面的代码中，定义了VGG网络结构，并在Fashion-MNIST数据集上进行了训练和测试。通过将图像尺寸从224缩小为96，可以看到模型的训练和推理速度可能会加快，但模型的性能可能会下降，因为图像的细节被压缩。这里使用了SGD优化器和交叉熵损失函数来训练模型，并输出了训练和测试的损失和准确率。最后输出了一个样例的预测结果，以查看模型的输出效果。请注意，实际训练过程可能需要更多的迭代次数和调整超参数以获得更好的性能。

4.请参考VGG论文 (Simonyan and Zisserman, 2014)中的表1构建其他常见模型，如VGG-16或VGG-19。

VGG-16:

<code>import torch

from torch import nn

from torchvision.transforms import ToTensor

from torchvision.datasets import FashionMNIST

from torch.utils.data import DataLoader

# 定义VGG块

def vgg_block(num_convs, in_channels, out_channels):

layers = []

for _ in range(num_convs):

layers.append(nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1))

layers.append(nn.ReLU())

in_channels = out_channels

layers.append(nn.MaxPool2d(kernel_size=2, stride=2))

return nn.Sequential(*layers)

# 定义VGG-16网络

class VGG16(nn.Module):

def __init__(self):

super(VGG16, self).__init__()

self.conv_blocks = nn.Sequential(

vgg_block(2, 3, 64),

vgg_block(2, 64, 128),

vgg_block(3, 128, 256),

vgg_block(3, 256, 512),

vgg_block(3, 512, 512)

)

self.fc_layers = nn.Sequential(

nn.Linear(512 * 7 * 7, 4096),

nn.ReLU(True),

nn.Dropout(),

nn.Linear(4096, 4096),

nn.ReLU(True),

nn.Dropout(),

nn.Linear(4096, 10)

)

def forward(self, x):

x = self.conv_blocks(x)

x = torch.flatten(x, start_dim=1)

x = self.fc_layers(x)

return x

# 将Fashion-MNIST图像转换为Tensor，并进行数据加载

transform = ToTensor()

train_dataset = FashionMNIST(root='./data', train=True, download=True, transform=transform)code>

test_dataset = FashionMNIST(root='./data', train=False, download=True, transform=transform)code>

# 创建数据加载器

batch_size = 128

train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

test_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False)

# 创建VGG-16模型实例

model = VGG16()

# 定义优化器和损失函数

optimizer = torch.optim.SGD(model.parameters(), lr=0.05)

criterion = nn.CrossEntropyLoss()

# 将模型移动到GPU（如果可用）

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

model.to(device)

# 训练模型

num_epochs = 10

for epoch in range(num_epochs):

model.train()

train_loss = 0.0

train_acc = 0.0

for images, labels in train_loader:

images = images.to(device)

labels = labels.to(device)

optimizer.zero_grad()

outputs = model(images)

loss = criterion(outputs, labels)

loss.backward()

optimizer.step()

train_loss += loss.item() * images.size(0)

_, preds = torch.max(outputs, 1)

train_acc += torch.sum(preds == labels.data).item()

train_loss /= len(train_loader.dataset)

train_acc /= len(train_loader.dataset)

# 在测试集上评估模型

model.eval()

test_loss = 0.0

test_acc = 0.0

with torch.no_grad():

for images, labels in test_loader:

images = images.to(device)

labels = labels.to(device)

outputs = model(images)

loss = criterion(outputs, labels)

test_loss += loss.item() * images.size(0)

_, preds = torch.max(outputs, 1)

test_acc += torch.sum(preds == labels.data).item()

test_loss /= len(test_loader.dataset)

test_acc /= len(test_loader.dataset)

print(f'Epoch {epoch+1}/{num_epochs}, Train Loss: {train_loss:.4f}, Train Accuracy: {train_acc:.4f}, '

f'Test Loss: {test_loss:.4f}, Test Accuracy: {test_acc:.4f}')

# 输出样例

print("Sample outputs:")

model.eval()

with torch.no_grad():

for images, labels in test_loader:

images = images.to(device)

labels = labels.to(device)

outputs = model(images)

_, preds = torch.max(outputs, 1)

for i in range(images.size(0)):

print(f"Predicted: {preds[i].item()}, Ground Truth: {labels[i].item()}")

break # 只输出一个批次的样例结果

以上代码展示了包含数据加载和训练的完整代码，使用VGG-16模型对Fashion-MNIST数据集进行了训练和测试。训练过程中，模型在每个epoch中进行了训练和验证，并输出了训练和验证集上的损失和准确率。最后，展示了一个样例输出，显示了模型在测试集上的预测结果。

VGG-19:

import torch

from torch import nn

from torchvision.transforms import ToTensor

from torchvision.datasets import FashionMNIST

from torch.utils.data import DataLoader

# 定义VGG块

def vgg_block(num_convs, in_channels, out_channels):

layers = []

for _ in range(num_convs):

layers.append(nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1))

layers.append(nn.ReLU())

in_channels = out_channels

layers.append(nn.MaxPool2d(kernel_size=2, stride=2))

return nn.Sequential(*layers)

# 定义VGG-19网络

class VGG19(nn.Module):

def __init__(self):

super(VGG19, self).__init__()

self.conv_blocks = nn.Sequential(

vgg_block(2, 3, 64),

vgg_block(2, 64, 128),

vgg_block(4, 128, 256),

vgg_block(4, 256, 512),

vgg_block(4, 512, 512)

)

self.fc_layers = nn.Sequential(

nn.Linear(512 * 7 * 7, 4096),

nn.ReLU(True),

nn.Dropout(),

nn.Linear(4096, 4096),

nn.ReLU(True),

nn.Dropout(),

nn.Linear(4096, 10)

)

def forward(self, x):

x = self.conv_blocks(x)

x = torch.flatten(x, start_dim=1)

x = self.fc_layers(x)

return x

# 将Fashion-MNIST图像转换为Tensor，并进行数据加载

transform = ToTensor()

train_dataset = FashionMNIST(root='./data', train=True, download=True, transform=transform)code>

test_dataset = FashionMNIST(root='./data', train=False, download=True, transform=transform)code>

# 创建数据加载器

batch_size = 128

train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)

test_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False)

# 创建VGG-19模型实例

model = VGG19()

# 定义优化器和损失函数

optimizer = torch.optim.SGD(model.parameters(), lr=0.05)

criterion = nn.CrossEntropyLoss()

# 将模型移动到GPU（如果可用）

device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')

model.to(device)

# 训练模型

num_epochs = 10

for epoch in range(num_epochs):

model.train()

train_loss = 0.0

train_acc = 0.0

for images, labels in train_loader:

images = images.to(device)

labels = labels.to(device)

optimizer.zero_grad()

outputs = model(images)

loss = criterion(outputs, labels)

loss.backward()

optimizer.step()

train_loss += loss.item() * images.size(0)

_, preds = torch.max(outputs, 1)

train_acc += torch.sum(preds == labels.data).item()

train_loss /= len(train_loader.dataset)

train_acc /= len(train_loader.dataset)

# 在测试集上评估模型

model.eval()

test_loss = 0.0

test_acc = 0.0

with torch.no_grad():

for images, labels in test_loader:

images = images.to(device)

labels = labels.to(device)

outputs = model(images)

loss = criterion(outputs, labels)

test_loss += loss.item() * images.size(0)

_, preds = torch.max(outputs, 1)

test_acc += torch.sum(preds == labels.data).item()

test_loss /= len(test_loader.dataset)

test_acc /= len(test_loader.dataset)

print(f'Epoch {epoch+1}/{num_epochs}, Train Loss: {train_loss:.4f}, Train Accuracy: {train_acc:.4f}, '

f'Test Loss: {test_loss:.4f}, Test Accuracy: {test_acc:.4f}')

# 输出样例

print("Sample outputs:")

model.eval()

with torch.no_grad():

for images, labels in test_loader:

images = images.to(device)

labels = labels.to(device)

outputs = model(images)

_, preds = torch.max(outputs, 1)

for i in range(images.size(0)):

print(f"Predicted: {preds[i].item()}, Ground Truth: {labels[i].item()}")

break # 只输出一个批次的样例结果