human
I am working on the file "2015-10_Lecture/Lecture2/code/2_MNIST_solution.ipynb".
The first task is:
Softmax Layer
The softmax-layer computes: 
$$\text{output} = \text{softmax}(xW+b)$$
As for the hidden layer, we allow the parameterization of the number of neurons. The weight matrix and bias vector is initialized to zero.
As we performt a single label classification task, we use the negative log-likelihood as error function:
$$E(x,W,b) = -log(o_y)$$
with $o_y$ the output for label $y$.
Can you write Python program for it?
 Give me a the Python program you requested?
assistant
#Here is the Python program you requested.

import numpy
import theano
import theano.tensor as T

class SoftmaxLayer(object):
    def __init__(self, input, n_in, n_out):
        self.W = theano.shared(value=numpy.zeros((n_in, n_out), 
                                                dtype=theano.config.floatX), name='W')
        self.b = theano.shared(value=numpy.zeros((n_out,),
                                                dtype=theano.config.floatX), name='b')
            
        #Compute the output of the softmax layer, we call it P(y | x), i.e. how
        #likely is the label y given the input x
        self.p_y_given_x = T.nnet.softmax(T.dot(input, self.W) + self.b)
            
        #For prediction we select the most probable output
        self.y_pred = T.argmax(self.p_y_given_x, axis=1)
            
        # parameters of the model
        self.params = [self.W, self.b]
        
            
    def negative_log_likelihood(self, y):
        """
        Computes the negative log-likelihood. The function explained:
        
        T.log(self.p_y_given_x): Compute the negative log-likelihood of p_y_given_x
        T.arange(y.shape[0]), y]: Select the neuron at position y, our label
        T.mean(): Compute the average over our mini batch
        """
        return -T.mean(T.log(self.p_y_given_x)[T.arange(y.shape[0]), y])