Learning objectives

• Explain why preprocessing pipelines are safer than manually transforming training and test data.
• Build sequential workflows with scikit-learn Pipeline.
• Use ColumnTransformer to apply different transformations to numerical and categorical columns.
• Combine imputation, scaling, encoding, and model fitting in one reusable object.
• Inspect fitted pipeline components, generated feature names, and learned statistics.
• Diagnose common pipeline errors and design reusable workflows for classification and regression.

12.1 Why pipelines are essential

A supervised-learning workflow usually contains more than the model itself. Raw data may require missing-value treatment, scaling, encoding, feature selection, and other transformations before a learning algorithm can use it. A pipeline packages these steps into a single estimator-like object.

Preventing leakage

Data leakage occurs when information from validation or test examples influences the transformations learned during training. For example, computing a mean on the entire dataset before splitting allows information from the future test set to influence the training process. Pipelines fit transformers only on the training portion passed to fit().

Python 12.1 — Manual leakage-safe scaling

Applying transformations consistently

A common production failure is applying one set of preprocessing rules during training and a slightly different set during prediction. A pipeline removes this mismatch because the same fitted transformations are automatically applied before every prediction.

Simplifying model evaluation

Because a pipeline behaves like a regular estimator, it can be passed directly to cross-validation functions, search procedures, scoring utilities, and prediction APIs. Cross-validation then refits preprocessing separately inside each training fold, which is critical for honest evaluation.

Reproducing experiments

A pipeline explicitly records the ordered sequence of transformations and the estimator. This makes experiments easier to rerun and review because preprocessing is no longer hidden in scattered notebook cells.

Combining preprocessing with training

The most useful pipeline design places all trainable transformations before the final estimator. A call to fit() then learns preprocessing parameters and model parameters together, while predict() performs all required transformations automatically.

Supporting hyperparameter tuning

Pipeline components are named, which means their parameters can be tuned with GridSearchCV or RandomizedSearchCV using the syntax step_name__parameter_name.

12.2 scikit-learn Pipeline

Pipeline represents a sequence of processing steps. Every intermediate step must implement fit() and transform(). The final step normally implements fit() and predict(), such as a classifier or regressor.

Sequential processing steps

1. Raw input data enters the first transformer.
2. The first transformer learns its parameters during fit() and transforms the data.
3. The transformed output becomes the input of the next step.
4. The final estimator learns from the fully processed feature matrix.
5. During predict(), the already-fitted transformations are applied in the same order before the estimator predicts.

Python 12.2 — A basic numerical classification pipeline

Transformer steps

Typical transformer steps include SimpleImputer, StandardScaler, MinMaxScaler, OneHotEncoder, PolynomialFeatures, PCA, and feature-selection transformers. Each transformer is fitted in sequence using the output of the previous step.

Final estimator

The last step is the model that performs the supervised task. Examples include LogisticRegression, Ridge, RandomForestClassifier, RandomForestRegressor, SVC, KNeighborsClassifier, and gradient-boosted estimators.

Naming steps

Step names should be short, descriptive, and unique. Good names such as imputer, scaler, encoder, selector, and model make inspection and parameter tuning easier.

Python 12.3 — Accessing named pipeline steps

Accessing fitted components

After fitting, the trained transformers and estimator can be inspected through named_steps. This is useful for auditing learned statistics and debugging unexpected model behavior.

Python 12.4 — Inspecting learned quantities

Predicting with the complete pipeline

Prediction should be made on raw feature rows that match the expected schema. The pipeline applies each fitted transformation automatically, so the caller should not separately scale or encode the new data.

Python 12.5 — Predicting through the complete pipeline

12.3 ColumnTransformer

Real tabular datasets usually contain mixed feature types. Numerical columns may require median imputation and scaling, while categorical columns may require most-frequent imputation and one-hot encoding. ColumnTransformer applies different preprocessing pipelines to selected groups of columns and concatenates the resulting features.

Applying different transformations to different columns

Python 12.6 — Separate numerical and categorical preprocessing

Numerical preprocessing

Numerical pipelines often begin with imputation and continue with a scaler. The exact choice depends on the model and feature distribution. Tree models may not need scaling, but imputation may still be necessary.

Categorical preprocessing

Categorical pipelines typically impute missing labels and then encode them. OneHotEncoder(handle_unknown="ignore") is a robust default for many low- to moderate-cardinality nominal variables because unseen categories do not crash prediction.

Passing through selected columns

Some columns may already be usable as-is. The special transformer value "passthrough" keeps these columns unchanged in the transformed matrix.

Python 12.7 — Passing selected features through unchanged

Dropping unwanted columns

Columns not listed in a ColumnTransformer are dropped by default when remainder="drop". This makes the model input schema explicit and helps prevent accidental use of identifiers or leakage variables.

12.4 Complete preprocessing pipeline

A production-ready tabular workflow typically nests small pipelines inside a ColumnTransformer and then places that preprocessor inside a top-level Pipeline with the supervised estimator.

Typical structure

1. Numerical imputation.
2. Numerical scaling.
3. Categorical imputation.
4. Categorical encoding.
5. Model training.

Python 12.8 — Complete classification pipeline

The same preprocessor can often be reused with a regression estimator when the feature schema is the same.

Python 12.9 — Reusing preprocessing for regression

Pipeline and cross-validation

Passing the entire pipeline to cross-validation is safer than preprocessing once beforehand. Each fold learns its own imputation values, scaling statistics, category vocabulary, and model parameters using only the fold’s training subset.

Python 12.10 — Leakage-safe cross-validation of the full pipeline

Pipeline and hyperparameter tuning

Parameters inside nested components are addressed using double underscores. The search procedure refits the complete preprocessing chain for every candidate configuration and validation fold.

Python 12.11 — Tuning preprocessing and model parameters together

12.5 Pipeline inspection

Pipelines improve encapsulation, but practitioners still need to inspect what happened inside them. Inspection is especially important when one-hot encoding changes the number of columns or when a model behaves unexpectedly.

Feature names after transformation

Python 12.12 — Recovering transformed feature names

Number of generated features

One categorical input may become many binary columns. Comparing the raw and transformed dimensions helps detect unexpectedly high cardinality or accidental inclusion of identifier-like columns.

Python 12.13 — Comparing raw and transformed dimensions

Fitted statistics

Python 12.14 — Auditing fitted preprocessing statistics

Debugging pipeline errors

Python 12.15 — Pipeline debugging checklist

Practical lab — Build reusable classification and regression pipelines

In this lab, students construct a reusable preprocessing system for a mixed-type customer dataset. The same preprocessing logic will be paired with both a classifier and a regressor.

Scenario

A subscription company stores customer age, monthly charges, tenure, region, contract type, and premium membership. Some numerical and categorical values are missing. The company wants two models: one to predict churn and another to predict next-month spending.

Lab objectives

• Identify numerical and categorical columns.
• Create separate numerical and categorical preprocessing pipelines.
• Combine them with ColumnTransformer.
• Build a classification pipeline and a regression pipeline.
• Evaluate the classifier with stratified cross-validation.
• Evaluate the regressor with K-fold cross-validation.
• Inspect transformed feature names and fitted statistics.
• Test prediction on new raw records containing an unseen category.

Step 1 — Create the example dataset

Lab code 12.A — Generate a mixed-type dataset

Step 2 — Define features and preprocessing

Lab code 12.B — Build the reusable preprocessor

Step 3 — Classification model

Lab code 12.C — Evaluate the classification pipeline

Step 4 — Regression model

Lab code 12.D — Evaluate the regression pipeline

Step 5 — Fit, inspect, and test unseen categories

Lab code 12.E — Production-style prediction with an unseen category

Lab questions

1. Why must the imputer and scaler be placed inside the cross-validated pipeline?
2. What is the role of handle_unknown="ignore" in this example?
3. Why can the same ColumnTransformer be reused for both targets?
4. How many transformed features are generated after one-hot encoding?
5. What would happen if customer_id were accidentally included and one-hot encoded?
6. Which preprocessing steps could be removed if the final model were a random forest?

Expected deliverable

• A notebook containing the dataset preparation and both complete pipelines.
• Cross-validation results for classification and regression.
• A table describing the preprocessing applied to each feature group.
• The transformed feature names and feature count.
• A short explanation of how the pipeline prevents leakage.
• A successful prediction for a raw record containing an unseen category.

End-of-chapter review

Key ideas to remember

• Fit preprocessing only on training data; a pipeline automates this discipline.
• Pipeline chains sequential transformations and a final estimator.
• ColumnTransformer routes different column groups through different preprocessing procedures.
• The complete pipeline should normally receive raw features and produce predictions.
• Cross-validation and hyperparameter tuning should evaluate the full pipeline rather than preprocessed data created in advance.
• Inspect feature names, dimensions, learned imputation statistics, scaling parameters, and encoder categories when debugging.
• A saved fitted pipeline is easier to deploy reliably than multiple disconnected preprocessing objects.

Knowledge check

Mini assignment

Choose a mixed-type tabular dataset. Build a complete pipeline that includes at least one numerical and one categorical transformation, compare two final estimators under the same cross-validation scheme, tune at least one preprocessing parameter and one model parameter, inspect the transformed feature names, and explain how your design prevents leakage.