human
I am working on the file "03-neural-networks/lecture.ipynb".
The first task is:
Ingredients for backpropagation
Implement two simple modules that know how to compute their "local" gradients. Then build up
a more complicated module that contains those two. We can treat each successive module
as a black-box that just "magically" knows how to analytically compute the gradients.
Below we implement $f(x, y, z) = (x+y)z$ and compute the gradient.
Can you write Python program for it?
 Write me a Python program to compute their "local" gradients?
assistant
#Here is Python program to compute their "local" gradients.

class Multiply:
    def forward(self, x, y):
        self.x = x
        self.y = y
        return x * y
    
    def backward(self, dLdz):
        dzdx = self.y
        dLdx = dLdz * dzdx
        
        dzdy = self.x
        dLdy = dLdz * dzdy
        return [dLdx, dLdy]

class Add:
    def forward(self, x, y):
        self.x = x
        self.y = y
        return x + y
        
    def backward(self, dLdz):
        dzdy = 1
        dzdx = 1
        return [dLdz * dzdy, dLdz * dzdx]

def f_with_gradients(x, y, z):
    # create our operators
    q = Add()
    f = Multiply()
    
    # feed inputs into the summer first, then do multiplication
    # this builds our computational graph
    q_out = q.forward(x, y)
    f_out = f.forward(q_out, z)
    
    # this one is somehow weird ... but hey.
    # step backwards through our graph to compute the gradients
    grad_f = f.backward(1.)
    grad_q = q.backward(grad_f[0])

    # sort our gradients so we have [df/dx, df/dy, df/dz]
    gradients = [grad_q[0], grad_q[1], grad_f[1]]
    return f_out, gradients

f_with_gradients(-2, 5, -4)