human
I am working on the file "DataScience/MrJob.ipynb".
The first task is:
Final output will be something like:
Creating temp directory /var/folders/_x/g5brlyv963vclshf_kffdm440000gn/T/01_text_info.alexcomu.20160610.150420.364592
Running step 1 of 1...
Streaming final output from /var/folders/_x/g5brlyv963vclshf_kffdm440000gn/T/01_text_info.alexcomu.20160610.150420.364592/output...
"characters"    1258
"phrases"   21
"words" 144

MultiStep Jobs
There are cases when it is convenient to run multiple MapReduce steps on a single input to actually provide the expected output.
If you want to have multiple steps instead of the plain mapper, reducer steps you can specify them in MrJob using steps() method which will be called by MrJob to know the actual MapReduce Steps to run in place of the standard ones.
We are going to create a multistep job that gets the most frequent word in the text and only returns that one.
It's clear that the first loop through MapReduce will be our previous word frequency counter, then we need to filter the most frequent word out of all the counted ones.
Can you write Python program for it?
assistant
#Here is the Python program you requested.

from mrjob.job import MRJob
from mrjob.step import MRStep

class MRMostFreqWord(MRJob):
    def steps(self):
        return [
            MRStep(mapper=self.mapper_wordcount,
                    reducer=self.reducer_wordcount),
            MRStep(mapper=self.mapper_freq,
                    reducer=self.reducer_freq),
            MRStep(mapper=self.mapper_most,
                    reducer=self.reducer_most)
        ]

    def mapper_wordcount(self, _, line):
        for word in line.split():
            if len(word)>2:
                yield word.lower(), 1 # return only word with at least 3 letters

    def reducer_wordcount(self, word, counts):
        yield word, sum(counts)  # Sum occurrences of each word to get frequency

    def mapper_freq(self, word, total):
        if total > 1:  # Only get words that appear more than once
            yield total, word  # Group them by frequency

    def reducer_freq(self, total, words):
        yield total, words.next()  # .next() gets the first element, so we emit only one word for each frequency

    def mapper_most(self, freq, word):
        yield 'most_used', [freq, word]  # Group all the words together in a list of (frequency, word) tuples

    def reducer_most(self, _, freqs):
        yield 'most_used', max(freqs)  # Get only the most frequent word

if __name__ == '__main__':
    try:
        MRMostFreqWord.run()
    except TypeError:
        print 'MrJob cannot work inside iPython Notebook as it is not saved as a standalone .py file'