human
I am working on the file "FADL1/L7CA_lesson7-cifar10.ipynb".
The first task is:
The idea is, if we have some inputs coming in and a function trying to predict how much the error is, then add on another prediction of error at that new stage, and on and on, then each time we're zooming in closer and closer to the correct answer -- ie: we've gotten to a certain point but there's still an error, a residual, so let's create a model that predicts that residual and add that onto our previous model, and another model that predicts that residual, and adds that on, and etc. If you keep doing that over and over, you should get closer and closer to our answer.
This is based on the theory of Boosting. By specifying return x + super().forward(x) as the thing we're trying to calculate, then we're kind of getting boosting for free.
Note that here, only one convolution is done in the ResNet block. Actual standard ResNet blocks use two convolutions before adding back onto the input.
Note also that the first layer in every block is a standard Conv layer w/ a stride of two, not a Res layer. This is a bottleneck layer. From time to time we change the geometry in a ResNet model. Actual ResNets don't use a standard Conv layer for bottlenecks.
but that'll be covered in Part 2 of this course.
Still, this simplified ResNet gets up to 82% accuracy (incl. overfitting).
8. ResNet2
We can make the Resnet even bigger
Can you write Python code for it?
 Give me a the Python code you requested?
assistant
#Here is the Python code you requested.

class Resnet2(nn.Module):
    def __init__(self, layers, c, p=0.5):
        super().__init__()
        self.conv1 = BnLayer(3, 16, stride=1, kernel_size=7)
        self.layers = nn.ModuleList([BnLayer(layers[i], layers[i+1])
            for i in range(len(layers) - 1)])
        self.layers2 = nn.ModuleList([ResnetLayer(layers[i+1], layers[i + 1], 1)
            for i in range(len(layers) - 1)])
        self.layers3 = nn.ModuleList([ResnetLayer(layers[i+1], layers[i + 1], 1)
            for i in range(len(layers) - 1)])
        self.out = nn.Linear(layers[-1], c)
        self.drop = nn.Dropout(p) # dropout added
    
    def forward(self, x):
        x = self.conv1(x)
        for λ,λ2,λ3 in zip(self.layers, self.layers2, self.layers3):
            x = λ3(λ2(λ(x)))
        x = F.adaptive_max_pool2d(x, 1)
        x = x.view(x.size(0), -1)
        x = self.drop(x)
        return F.log_softmax(self.out(x), dim=-1)