human
I am working on the file "LeNet5.ipynb".
The first task is:
Momentum
Momentum is a varient of Vanilla SGD that helps accelerate SGD in the relevant direction and dampens oscillations. It does this by adding a fraction $\gamma$ of the update vector of the past time step to the current update vector. Nevertheless, like Vanilla SGD it still has difficulty escaping a saddle point. 
The update rule is given by:
<h5 align="center">
$\begin{align}
v_{t+1} &= \gamma v_t + \eta \nabla C\\
x_{t+1} &=  x_t- v_{t+1}
\end{align}$
</h5>

where $v_t$ can be considered as the velocity.
Can you write Python program for it?
 ... Give me a the Python program you requested?
assistant
#Here is the Python program you requested.

def momentum(l_rate, parameters, grads, momentum=0.9):
    
    def update_rule(param, velocity, df):
        v_next = momentum * velocity - l_rate * df
        updates = (param, param+v_next), (velocity, v_next)

        return updates

    assert momentum <=1 and momentum >= 0

    velocities = [theano.shared(name='v_%s' % param,
                                value=param.get_value() * 0., 
                                broadcastable=param.broadcastable) 
                  for param in parameters]

    updates = []
    for p, v, g in zip(parameters, velocities, grads):
        param_updates, vel_updates = update_rule(p, v, g)
        updates.append(param_updates)
        updates.append(vel_updates)

    return updates