Model Ensemble

This is text I want to write

class tubesml.stacker.Stacker(estimators, final_estimator, cv, lay1_kwargs=None, passthrough=False, verbose=False, bypass_importances=False)

Wrapper for stacking several estimators with a meta estimator. Each estimator creates out of fold predictions on the entire dataset via a KFold and it is then re-fitted against the full dataset. The predictions are generated by tubesml.CrossValidate and thus can account for early stopping and other options.

The meta estimator trains on the predicitions made by the estimators. It is possible to train the meta estimator on other features of the initial dataset.

Parameters:

  • estimators – list of tuples (name, estimator) (including pipelines). These estimators will generate the first layer of predictions.

  • final_estimator – estimator with a fit and a predict (or predict_proba) methods. This estimator will create the final prediction. Alternatively, it can be just passed the string blend to not fit any final estimator and just take an average of the predictions at level 1.

  • cv – KFold (or similar) generator. The CV scheme to use to generate the first layer of predictions.

  • lay1_kwargs – (optional) dict, Dictionary of settings for the first layer predictions to be passed to tubesml.CrossValidate. The keys of the dictionary refer to the names in the list of estimators.

  • passthrough –

    bool or list, default=False. If True, all the features used to train the first layer of estimators will be used

    by the final_estimator.

    If list, only the listed features will be used. If the input is incorrect, it will be set to False

  • verbose – bool, default=False. If True, it warns the user if the correlation of the first layer of predictions is higher than 0.9.

Attributes:
meta_importances_pandas DataFrame with the feature importances (or the coefficients) of the

final estimator. Note: this estimator doesn’t have a coef_ or feature_importances_ attribute, it won’t thus work with methods that leverage these attritbutes.

corr_pandas DataFrame.

Returns the correlation between the first set of estimator’s predictions.

fit(X, y)

This method uses tubesml.cv_score to create out of fold predictions from each of the estimators provided in estimators.

Secondly, it fits the final_estimator on a dataset that contains these predictions and any feature specified by passthrough. Each of the estimators is then refit on the entire dataset

If an estimator is in the first layer of estimators was trained with early stopping, in the refit it will be trained on a number of iterations equal to the mean number across the folds used to generate the first layer of predictions. Be sure that the early stopping attribute of the estimator is early_stopping_round.

If verbose=True the user will be warned if any of the predictions are correlated more than 0.9. The corr_ attribute is created by this method

Parameters:

  • X – pandas DataFrame of shape (n_samples, n_features) The training input samples.

  • y – array-like of shape (n_samples,) or (n_samples, n_outputs), Not used The target values (class labels) as integers or strings.

predict(X, y=None)

Method to generate the final predictions. First, it generates the meta dataset with the predictions by the estimators.

If any of them was generated by a predict_proba method, it will be done again, otherwise it uses the predict method of those estimators.

The final prediction is generated by using the predict method of the final_estimator.

Parameters:

  • X – pandas DataFrame of shape (n_samples, n_features) The training input samples.

  • y – array-like of shape (n_samples,) or (n_samples, n_outputs), Not used The target values (class labels) as integers or strings.

Return preds:

ndarray of shape (n_samples,) or (n_samples, n_output) Predicted targets.

predict_proba(X, y=None)

Method to generate the final predictions. First, it generates the meta dataset with the predictions by the estimators.

If any of them was generated by a predict_proba method, it will be done again, otherwise it uses the predict method of those estimators.

The final prediction is generated by using the predict_proba method of the final_estimator.

Parameters:

  • X – pandas DataFrame of shape (n_samples, n_features) The training input samples.

  • y – array-like of shape (n_samples,) or (n_samples, n_outputs), Not used The target values (class labels) as integers or strings.

Return preds:

ndarray of shape (n_samples, n_classes) or list of ndarray of shape (n_output,) The class probabilities of the input samples.

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → Stacker

Request metadata passed to the score method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

  • True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

  • False: metadata is not requested and the meta-estimator will not pass it to score.

  • None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

  • str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

Returns

selfobject

The updated object.