human
I am working on the file "Linear_SVM.ipynb".
The first task is:
Explain how the huberized hinge loss relates to the regular hinge loss and to the misclassification error loss. 
Well, Huberized Hinge Loss is a smooth version of Hinge Loss that is differentiable everywhere. As can be seen in the plot, Hinge Loss and Huberized Hinge Loss give loss even when the function output is less than 1. Misclassification Error Loss only acts after the function output falls below 0.
Write a mathematical proof that the huberized hinge loss is differentiable (hint: the huberized hinge loss is defined piecewise).
Assuming yt = $x$ and function $f$ as the huberized hinge loss,
$$
f(x) = \left{\begin{aligned}
&0 &: x > 1+h\
&\frac{(1+h-x)^2}{4h} &: 1-h \leq x \leq 1+h\
&1-x &: x < 1-h
\end{aligned}
\right.$$
Piecewise differentiation of $f$,
$$
f'(x) = \left{\begin{aligned}
&0 &: x > 1+h\
&-\frac{2(1+h-x)}{4h} &: 1-h \leq x \leq 1+h\
&-1 &: x < 1-h
\end{aligned}
\right.$$
Clearly, $f$ is differentiable on each piece.
Let's check $f$'s differentiability on the kinks using $f'(x)$ $= \lim_{p\to x}\frac{f(p)}{p}$.
For $1+h$,
\begin{align}
f'(1+h) =& \lim_{p\to 1+h}\frac{f(p)}{p}
\end{align}
Right limit is,
\begin{align}
f'(1+h) =& \lim_{p\to (1+h)^+}\frac{0}{1+h}
= 0
\end{align}
Left limit is,
\begin{align}
f'(1+h) =& \lim_{p\to (1+h)^-}\frac{\frac{(1+h-(1+h))^2}{4h}}{1+h}
= 0
\end{align}
Clearly, Right limit = Left limit at $1+h$. Therefore, the function is differentiable at this point.
For $1-h$ right limit is,
\begin{align}
f'(1-h) =& \lim_{p\to (1-h)^+}\frac{\frac{(1+h-(1-h))^2}{4h}}{1-h} = \frac{h}{1-h} 
\end{align}
Left limit is,
\begin{align}
f'(1-h) =& \lim_{p\to (1-h)^-}\frac{1-(1-h)}{1-h} = \frac{h}{1-h} 
\end{align}
Clearly, Right limit = Left limit at $1-h$. Therefore, the function is differentiable at this point.
This proves that the function is differentiable at all points.
Write the analytic expression(s) of the gradient of the huberized hinge loss.  
Analytic expression(s) of the gradient of the huberized hinge loss is,
$$
l_{huber-hinge}'(yt) = \left{\begin{aligned}
&0 &: yt >...
Can you write Python program for it?

=====
Give me a the Python program you requested?
assistant
#Here is the Python program you requested.

def compute_obj(x, y, w, C=1.0, h=0.5):
    loss = np.vectorize(huberizedHingeLoss, excluded=['h'])
    return np.dot(w, w) + (C/float(x.shape[0]))*sum(loss(y*np.dot(x,w), h))

def compute_grad(x, y, w, C=1.0, h=0.5):
    p = y*np.dot(x, w)
    gradW = np.zeros(w.shape[0], dtype=float)
    
    def gradHuberHinge(i, j):
        if p[i] > 1+h:
            return 0
        elif abs(1-p[i]) <= h:
            return ((1+h-p[i])/(2*h))*(-y[i]*x[i][j])
        else:
            return (-y[i]*x[i][j])
        
    for j in range(w.shape[0]):
        sum_over_i = 0.0
        for i in range(x.shape[0]):
            sum_over_i += gradHuberHinge(i,j)
        gradW[j] = 2*w[j] + (C/float(x.shape[0]))*sum_over_i
    return gradW

def add_bias_column(x):
    return np.append(x, np.ones(x.shape[0]).reshape(x.shape[0],1), axis=1)

n_samples = 1000
n_features = 30
x, y = make_blobs(n_samples, n_features, centers=2)
y[y==0] = -1
x = add_bias_column(x)

w = np.zeros(n_features+1)

compute_obj(x, y, w)

compute_grad(x, y, w)