human
Write me a the Python program you requested given the following:
1. (1.7)
Implement a MDP that solves the question 2 of HW4 from [AI-edX]
assistant
#Here is the Python program you requested.
class LinearMDP(GridMDP):
    """A two-dimensional grid MDP, as in [Figure 17.1].  All you have to do is
    specify the grid as a list of lists of rewards; use None for an obstacle
    (unreachable state).  Also, you should specify the terminal states.
    An action is an (x, y) unit vector; e.g. (1, 0) means move east."""

    def __init__(self, grid, terminals, init=(0, 0), gamma=.9):
        GridMDP.__init__(self, grid=grid, terminals=terminals, init=init, gamma=gamma)

    def T(self, state, action):
        """
        This function must return a list of tuples (p, state') where p is the probability 
        of going to state'
        """
        # Your code here #

def calculate_v_star(rew_a, gamma):
    '''
    This function must create an instance of LinearMDP that corresponds to the 
    MDP in question 2 of HW4 and use it to calculate the expected reward for each state.
    The function receives as parameter the reward for state 'a', which in the example
    corresponds to 10, but here is variable, and the value of gamma. 
    The reward for state 'e' is 1. The function must return a dictionary with the V* values. 
    '''
    # Your code here #
    mdp = 
    v_star =
    ##################
    return v_star