human
I am working on the file "A Noob's guide to text classification [Unrendered].ipynb".
The first task is:
Now let us put it all together. 
Gensim
Note that corpus that we loaded using <b><i>load_files</i></b> above resides fully in memory, as a plain Python Bunch. When the input corpus becomes very big with millions of documents , storing all of them in RAM may be a bad design decision and could crash the system. Gensim overcomes this by using a technique called streaming. While streaming a document is loaded and processed in memory one by one. Although the output is the same as for the plain Python bunch, the corpus is now much more memory friendly, because at most one vector resides in RAM at a time. The implementation of a gensim corpus class is as shown below:
Can you write Python program for it?

What is a the Python program you requested?
assistant
#Here is the Python program you requested.

# to print the gensim logging in ipython notebook
import logging
logging.basicConfig(format='%(levelname)s : %(message)s', level=logging.INFO)
logging.root.level = logging.INFO  # ipython sometimes messes up the logging setup; restore

import gensim
import os
from os.path import join
from gensim import utils
import datetime, re, sys
from nltk.tokenize import RegexpTokenizer
from gensim import similarities,models, matutils
from gensim.similarities import MatrixSimilarity, SparseMatrixSimilarity, Similarity
from gensim import corpora
tokenizer = RegexpTokenizer(r'[a-zA-Z0-9]+')

# Tokenizer
def text_tokenize(text):        
    tokens = tokenizer.tokenize(text.lower())
    # return the transformed corpus
    out=[]
    for token in tokens:
        # we filter out tokens that exist in stopwords list 
        if token not in stopwords_list:
            out.append(token)
            
    return out  # returns a list

class MyCorpus(gensim.corpora.TextCorpus):
    def get_texts(self):
        for filename in self.input:
            yield text_tokenize(open(filename).read())