Data Cleaning

In this section, we can find the classes and methods typically used to clean the data up for transformations and models at a later stage in a pipeline.

class tubesml.clean.DfImputer(imputer_type='simple', strategy='mean', fill_value=None, add_indicator=False, n_neighbors=5, weights='uniform')

Just a wrapper for the SimpleImputer that keeps the dataframe structure.

Inherits from BaseTransformer.

Parameters:

  • strategy – str, the strategy to impute the missing values, default “mean”. Allowed values: “mean”, “median”, “most_frequent”, “constant”

  • fill_value – value to use to impute the missing values when the strategy is “constant”. It is ignored by any other strategy

  • add_indicator – bool, default=False. If True, a new column with binary values is created whenever missing values are found when the fit method is called. The column will be called missing_<column_name>

Attributes:
statistics_pandas Series. The statistics per column, depending on the strategy chosen.

The index of the series is the columns attribute of the input dataframe.

impsklearn.impute.SimpleImputer

Core transformer. Its fit and transform methods are used here.

fit(X, y=None)

Method to train the imputer.

It also reset the columns attribute

Parameters:

  • X – pandas DataFrame of shape (n_samples, n_features) The training input samples.

  • y – array-like of shape (n_samples,) or (n_samples, n_outputs), Not used The target values (class labels) as integers or strings.

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → DfImputer

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns

selfobject

The updated object.

transform(X, y=None)

Method to transform the input data

It populates the columns attribute with the columns of the output data

Parameters:

  • X – pandas DataFrame of shape (n_samples, n_features) The input samples.

  • y – array-like of shape (n_samples,) or (n_samples, n_outputs), Not used The target values (class labels) as integers or strings.

Returns:

pandas DataFrame with no missing values