human
I am working on the file "RNNLM.ipynb".
The first task is:
Building the RNN
For a general overview of RNNs take a look at first part of the tutorial.

Let's get concrete and see what the RNN for our language model looks like. The input $x$ will be a sequence of words (just like the example printed above) and each $x_t$ is a single word. But there's one more thing: Because of how matrix multiplication works we can't simply use a word index (like 36) as an input. Instead, we represent each word as a one-hot vector of size vocabulary_size. For example, the word with index 36 would be the vector of all 0's and a 1 at position 36. So, each $x_t$ will become a vector, and $x$ will be a matrix, with each row representing a word. We'll perform this transformation in our Neural Network code instead of doing it in the pre-processing. The output of our network $o$ has a similar format. Each $o_t$ is a vector of vocabulary_size elements, and each element represents the probability of that word being the next word in the sentence.
Let's recap the equations for the RNN from the first part of the tutorial:
$
\begin{aligned}
s_t &= \tanh(Ux_t + Ws_{t-1}) \
o_t &= \mathrm{softmax}(Vs_t)
\end{aligned}
$
I always find it useful to write down the dimensions of the matrices and vectors. Let's assume we pick a vocabulary size $C = 8000$ and a hidden layer size $H = 100$. You can think of the hidden layer size as the "memory" of our network. Making it bigger allows us to learn more complex patterns, but also results in additional computation. Then we have:
$
\begin{aligned}
x_t & \in \mathbb{R}^{8000} \
o_t & \in \mathbb{R}^{8000} \
s_t & \in \mathbb{R}^{100} \
U & \in \mathbb{R}^{100 \times 8000} \
V & \in \mathbb{R}^{8000 \times 100} \
W & \in \mathbb{R}^{100 \times 100} \
\end{aligned}
$
This is valuable information. Remember that $U,V$ and $W$ are the parameters of our network we want to learn from data. Thus, we need to learn a total of $2HC + H^2$ parameters. In the case of $C=8000$ and $H=100$ that's 1,610,000.  The dimensions also tell us the bottleneck of our model. Note that...
Can you write Python code for it?

=====
Give me a Python code to find it useful to write down the dimensions of the matrices and vectors?
assistant
#Here is Python code to find it useful to write down the dimensions of the matrices and vectors.

class RNNNumpy:
    
    def __init__(self, word_dim, hidden_dim=100, bptt_truncate=4):
        # Assign instance variables
        self.word_dim = word_dim
        self.hidden_dim = hidden_dim
        self.bptt_truncate = bptt_truncate
        # Randomly initialize the network parameters
        self.U = np.random.uniform(-np.sqrt(1./word_dim), np.sqrt(1./word_dim), (hidden_dim, word_dim))
        self.V = np.random.uniform(-np.sqrt(1./hidden_dim), np.sqrt(1./hidden_dim), (word_dim, hidden_dim))
        self.W = np.random.uniform(-np.sqrt(1./hidden_dim), np.sqrt(1./hidden_dim), (hidden_dim, hidden_dim))