human
2.2 Weight Initialization Methods

Gaussian (Normal) Distribution with a standard deviation $\sigma = 0.01$

Yann LeCun's Weight Initialization Method: Yann LeCun states that if the data is been normalized and tanh_lecun is been used, then the weights could be initialized from a distribution having a mean $\mu = 0$ and a standard deviation $\sigma = {\frac{1}{\sqrt{m}}}$ where $m$ is the number of inputs to a node.

Xavier / Glorot Weight Initialization Method: If $m$ is the number of fan-in nodes and $n$ is the number of fan-out nodes, then the weights could be initialized choosing a random value from a Uniform Distribution in the range $$U \sim [\,{\sqrt{\frac{6}{m + n}}}, {\sqrt{\frac{6}{m - n}}}]\,$$

Probability Density Function (Uniform Distribution)
$$P(x) = \frac{1}{b - a}$$
Probability Density Function (Normal Distribution)
$$P(x) = \frac{1}{\sqrt{2\pi\sigma^2}}e^{-\frac{(x - \mu)^2}{2\sigma^2}}$$
In order to randomize weights around 0, we pick out random values from the said probability density function.

=====
What is a the Python program you requested?
assistant
#Here is the Python program you requested.
# a probability density function for a Normal Distribution
def pdf_normal(x, mu = 0.0, sigma = 1.0):
    c = 2 * np.power(sigma, 2)
    return (1.0 / np.sqrt(np.pi * c)) * np.exp(-(np.power(x - mu, 2) / c))

# plotting a Uniform Distribution
def plot_uniform(low = 0.0, high = 1.0, bins = 100, size = 1000, title = None):
    range      = np.random.uniform(low, high, size)
    _, bins, _ = plt.hist(range, bins, normed = True)
    plt.legend(['$U \sim [\,%.2f, %.2f]\,$' % (low, high)])
    
    if title is not None:
        plt.title(title)
        
    plt.show()

# a test case to plot a Uniform Distribution
plot_uniform()

# plotting a Normal Distribution and its PDF
def plot_normal(mu = 0.0, sigma = 1.0, bins = 100, size = 1000, title = None):
    range      = np.random.normal(mu, sigma, size)
    _, bins, _ = plt.hist(range, bins, normed = True)
    plt.plot(bins, pdf_normal(bins, mu, sigma), lw = 2, color = 'r')
    
    if title is not None:
        plt.title(title)
        
    plt.legend(['$\mu = %0.2f, \sigma = %0.2f$' % (mu, sigma)])
    
    plt.show()

# a test case to plot a Normal Distribution and its PDF
plot_normal()

# Our Artificial Neural Network class
class ANN:
    # available weight initialization methods
    RANDN          = 1
    LECUN_UNIFORM  = 2
    LECUN_NORMAL   = 3
    XAVIER_UNIFORM = 4
    XAVIER_NORMAL  = 5
    
    # available activation functions 
    IDENTITY       = identity
    CLOGLOG        = cloglog
    RELU           = relu
    SIGMOID        = sigmoid
    TANH           = tanh
    TANH_LECUN     = tanh_lecun
    HEAVISIDE      = heaviside
    ABSOLUTE       = absolute
    
    '''
    @arguments
        sizes    - a list containing the number of neurons in each layer inclusive of the input and output layer.
        wimethod - the initialization method for initializating weights and biases.
    '''    
    def __init__(self, sizes, wimethod = RANDN, debug = False):
        self.log      = Log(debug) # a logger for debugging
        
        self.nlayers  = len(sizes) #...