human
Here else could we do given the following:
4 Linear regression
Using numpy, fit and evaluate the linear model $$\hat{w}, \hat{b} = \operatorname*{arg min}_{w,b} \| Xw + b - y \|_2^2.$$
Please define a class LinearRegression with two methods:
1. fit learn the parameters $w$ and $b$ of the model given the training examples.
2. predict gives the estimated number of likes of a post / tweet. That will be used to evaluate the model on the testing set.
To evaluate the classifier, create an accuracy(y_pred, y_true) function which computes the mean squared error $\frac1n \| \hat{y} - y \|_2^2$.
Hint: you may want to use the function scipy.sparse.linalg.spsolve().
If solve and spsolve tells you that your matrix is singular, please read this good comment. Potential solutions:
1. Is there any post / tweet without any word from the vocabulary ? I.e. a row of $X$ made only of zeroes. If yes, remove this row or enlarge the vocabulary.
2. Identify and remove redundant features, i.e. words, who are linear combinations of others.
3. What else could we do ?
assistant
#Here else could we do .
import scipy.sparse

class LinearRegression(object):
    
    def predict(self, X):
        """Return the predicted class given the features."""
        return X.dot(self.w) + self.b
    
    def fit(self, X, y):
        """Learn the model's parameters given the training data, the closed-form way."""
        n, d = X.shape
        self.b = y.mean()
        A = X.T.dot(X)
        b = X.T.dot(y - self.b)
        #self.w = np.linalg.solve(A, b)
        self.w = scipy.sparse.linalg.spsolve(A, b)

def evaluate(y_pred, y_true):
    return np.linalg.norm(y_pred - y_true, ord=2)**2 / y_true.size

model = LinearRegression()
model.fit(X_train, y_train)
y_pred = model.predict(X_test)

mse = evaluate(y_pred, y_test)
print('mse: {:.4f}'.format(mse))