human
I am working on the file "Brains from Scratch - Machine Learning for dummies.ipynb".
The first task is:
Plant: Growth Habit and Leaf: Nectaries feature vectors seem to consists a high amount of missing data. Will dropping these columns affect the overall performance of our model? What about post imputation?
Well, nothing can be said for now.
Let's build a data set transfomer that can impute missing data within our data set. We can then pass such a transformer with a desired algorithm through our pipeline.
sklearn has a powerful TransformerMixin class that can be utilized to build our Imputation class. Our class shall be just a wrapper around fancyimpute (a library containing a wide range of imputation stratergies).
NOTE: sklearn has a class Imputer dedicated for imputing missing values. We'll be building an extension that implements strategies beyond ones provided by sklearn
Can you write Python code for it?
assistant
#Here about post imputation.

import numpy as np
from sklearn.base import TransformerMixin
from fancyimpute import SimpleFill, KNN, MICE

class DataFrameImputer(TransformerMixin):
    NUMERIC     = 0x000100
    CATEGORICAL = 0x000200
    '''
    Parameters
    ----------
    imputer - a fancyimpute class with desired parameters
    copy    - whether you'd like to perform an inplace imputation or return a copy
              default returns a copy
    impute  - NUMERIC, CATEGORICAL kinds of imputataion
              default is imputation on only numeric columns
    '''
    def __init__(self, imputer = SimpleFill(), impute = NUMERIC, copy = True):
        self.imputer = imputer
        self.copy    = copy
        self.impute  = impute

    def fit(self, *_):
        return self
    
    def transform(self, df):
        columns     = df.columns
        numeric     = list(column for column in columns if df[column].dtype != 'object')
        categorical = list(column for column in columns if df[column].dtype == 'object')
        frame       = df.copy() if self.copy else df
        
        frame[numeric]     = frame[numeric].fillna(np.nan)
        frame[categorical] = frame[categorical].fillna('NA')
        
        try:
            if self.impute is DataFrameImputer.NUMERIC:
                nparray        = frame[numeric].as_matrix()
                frame[numeric] = self.imputer.complete(nparray)
        except ValueError:
            pass

        return frame

imputer = DataFrameImputer(imputer = MICE(verbose = False), copy = False)
imputer.fit_transform(df)
df.sample(nsamples)