human
2.4 Learning as generic Optimization
Sometimes we don't want to implement the optimization by hand and would prefer a generic optimization algorithm. Let's make use of SciPy, which provides high-level algorithms for, e.g. optimization, statistics, interpolation, signal processing, sparse matrices, advanced linear algebra."
Please write code following the instructions in jupyter notebook style.

=====
What is a code to 2?
assistant
#Here is code to 2.

class RidgeRegressionOptimize(RidgeRegressionGradient):
    
    def __init__(self, alpha=0, method=None):
        """Here's a new hyper-parameter: the optimization algorithm."""
        super().__init__(alpha)
        self.method = method
    
    def fit(self, X, y):
        """Fitted with a general purpose optimization algorithm."""
        n, d = X.shape
        self.b = np.mean(y)
        
        # Objective and gradient w.r.t. the variable to be optimized.
        f = lambda w: self.loss(X, y, w)
        jac = lambda w: self.grad(X, y, w)
        
        # Solve the problem !
        from scipy.optimize import minimize
        w0 = np.random.normal(size=d)
        res = minimize(f, w0, method=self.method, jac=jac)
        self.w = res.x

models.append(evaluate(RidgeRegressionOptimize(alpha, method='Nelder-Mead')))
models.append(evaluate(RidgeRegressionOptimize(alpha, method='BFGS')))