human
I am working on the file "DAT236x Deep Learning Explained/Lab4_ConvolutionalNeuralNetwork.ipynb".
The first task is:
The first model we build is a simple convolution only network. Here we have two convolutional layers. Since, our task is to detect the 10 digits in the MNIST database, the output of the network should be a vector of length 10, 1 element corresponding to each output class. This is achieved by projecting the output of the last convolutional layer using a dense layer with the output being num_output_classes. We have seen this before with Logistic Regression and MLP where features were mapped to the number of classes in the final layer. Also, note that since we will be using the softmax operation that is combined with the cross entropy loss function during training (see a few cells below), the final dense layer has no activation function associated with it.
The following figure illustrates the model we are going to build. Note the parameters in the model below are to be experimented with. These are often called network hyperparameters. Increasing the filter shape leads to an increase in the number of model parameters, increases the compute time and helps the model better fit to the data. However, one runs the risk of overfitting. Typically, the number of filters in the deeper layers are more than the number of filters in the layers before them. We have chosen 8, 16 for the first and second layers, respectively. These hyperparameters should be experimented with during model building.
Can you write Python program for it?
 What is a Python program to compute time and helps the model better fit to the data?
assistant
#Here is Python program to compute time and helps the model better fit to the data.

# function to build model
# relu error:1.37
# sigmoid error: 2.69
# leaky relu error: 1.26
# tanh error: 1.4
# def create_model(features):
#     with C.layers.default_options(init=C.glorot_uniform(), activation=C.relu):
#     #with C.layers.default_options(init=C.glorot_uniform(), activation=C.sigmoid):
#     #with C.layers.default_options(init=C.glorot_uniform(), activation=C.leaky_relu):
#     #with C.layers.default_options(init=C.glorot_uniform(), activation=C.tanh):
#             h = features
#             h = C.layers.Convolution2D(filter_shape=(5,5), 
#                                        num_filters=8, 
#                                        strides=(2,2), 
#                                        pad=True, name='first_conv')(h)
#             h = C.layers.Convolution2D(filter_shape=(5,5), 
#                                        num_filters=16, 
#                                        strides=(2,2), 
#                                        pad=True, name='second_conv')(h)
#             r = C.layers.Dense(num_output_classes, activation=None, name='classify')(h)
#             return r

# average pool
# relu error:1.25    
# leak relu error: 1.17
# sigmoid error:3.89
# tanh erro:1.47
# def create_model(features):
#     with C.layers.default_options(init=C.glorot_uniform(), activation=C.leaky_relu):
#             h = features
#             h = C.layers.Convolution2D(filter_shape=(5,5), 
#                                        num_filters=8, 
#                                        strides=(1,1), 
#                                        pad=True, name='first_conv')(h)
#             h = C.layers.AveragePooling(filter_shape=(5,5),
#                                         strides=(2,2), name = 'first_avg_pool')(h)
#             h = C.layers.Convolution2D(filter_shape=(5,5), 
#                                        num_filters=16, 
#                                        strides=(1,1), 
#                                        pad=True, name='second_conv')(h)
#             h =...