Chapter overview

Categorical variables describe membership, type, status, level, region, device family, product group, or other qualitative concepts. Most machine-learning estimators cannot consume raw strings directly, so these variables must be transformed into numerical representations. The transformation is not merely a technical conversion: it encodes assumptions about similarity, order, rarity, and information content.

This chapter develops a practical framework for choosing among ordinal encoding, one-hot encoding, rare-category grouping, frequency encoding, target encoding, and hashing. Particular attention is given to leakage prevention, unknown categories at prediction time, sparse matrices, high cardinality, and production robustness.

Learning objectives

• Explain why raw category labels cannot usually be supplied directly to numerical learning algorithms.
• Distinguish nominal and ordinal categories and preserve their semantic meaning during encoding.
• Apply ordinal and one-hot encoding safely with scikit-learn.
• Control dimensionality by grouping infrequent categories and using frequency thresholds.
• Evaluate strategies for high-cardinality features, including frequency encoding, target encoding, and feature hashing.
• Prevent target leakage by fitting target-dependent encoders only within training folds.
• Configure encoders so unseen categories do not break production predictions.
• Construct separate numerical and categorical preprocessing procedures in a ColumnTransformer pipeline.

11.1 Why categorical variables require encoding

A categorical variable represents a finite set of labels rather than a physical numerical scale. Examples include payment method, city, product family, subscription plan, blood group, device type, and education level. A learning algorithm needs a numerical representation of these concepts before it can estimate distances, coefficients, split points, or gradients.

Machine learning algorithms expect numerical inputs

Many estimators operate on a numerical feature matrix X. Linear and logistic regression compute weighted sums; K-nearest neighbors and support vector machines depend on distances or inner products; neural networks use numerical tensors. A column containing values such as "Basic", "Premium", and "Enterprise" therefore requires a transformation before training.

Python example 11.1 — Inspect categorical columns

Category values do not automatically have numerical meaning

Replacing labels by arbitrary integers can accidentally introduce an order or a distance that does not exist. Coding {North=0, South=1, West=2} implies that West is numerically farther from North than South is, even though such geometry has no semantic basis. The encoding method must therefore match the variable’s meaning.

11.2 Ordinal encoding

Ordinal encoding is appropriate when categories possess a genuine, domain-defined order. The encoder maps each ordered category to an integer while preserving the ranking. The numerical spacing between consecutive codes should not automatically be interpreted as an equal physical distance.

Ordered categories

Examples include Low < Medium < High, Bronze < Silver < Gold, beginner < intermediate < advanced, or an agreement scale from strongly disagree to strongly agree. The order must be justified by domain semantics, not by alphabetical order or convenience.

Mapping categories to ordered numbers

Python example 11.2 — Explicit ordinal mapping with pandas

Python example 11.3 — OrdinalEncoder with a declared order

Appropriate examples

Risks of imposing false order

Ordinal encoding becomes dangerous when used for nominal variables. A model may treat the numerical codes as ordered or distance-bearing. In a linear model, moving from code 0 to code 1 has the same algebraic increment as moving from 1 to 2. In KNN, the codes influence distance directly. Tree models are less sensitive to numeric spacing but still split using thresholds, which can create artificial category groupings.

11.3 One-hot encoding

One-hot encoding represents each category using a separate binary indicator column. It is the standard default for low- to moderate-cardinality nominal variables because it avoids imposing an artificial order.

Binary columns for each category

Python example 11.4 — One-hot encoding with pandas

Python example 11.5 — OneHotEncoder in scikit-learn

Dropping a reference category

For some linear models, one category may be dropped to create a reference level. With a variable containing k categories, k−1 indicators are enough to reconstruct category membership when an intercept is present. This can reduce exact multicollinearity, although regularized estimators often work acceptably with all indicators.

Python example 11.6 — Drop a reference level

Unknown categories

Training data may contain North, South, and West, while a future request contains East. An encoder configured to raise an error can interrupt a prediction service. For robust inference, configure unknown-category behavior explicitly and test it before deployment.

Sparse matrices

One-hot matrices contain mostly zeros. Sparse representations store only nonzero entries, often reducing memory dramatically. This matters when many categorical variables or categories are present. Not every estimator accepts sparse matrices, so the downstream model and pipeline must be checked.

Python example 11.7 — Inspect sparse output

High-dimensional output

If a feature contains 10,000 unique categories, ordinary one-hot encoding can create up to 10,000 columns. The resulting feature space may consume memory, slow optimization, increase variance, and make interpretation difficult. Rare-category grouping or specialized high-cardinality encoders may be preferable.

11.4 Handling rare categories

A rare category appears only a small number of times or represents only a tiny fraction of the training data. Rare categories are statistically difficult: their estimated effects are uncertain, a model can memorize them, and future datasets may contain new variants that were absent during training.

Grouping infrequent categories

A common strategy replaces infrequent labels with a shared category such as "Other" or "Rare". The grouping rule must be learned from the training data and then applied unchanged to validation, test, and production data.

Python example 11.8 — Group categories using a frequency threshold

Minimum frequency thresholds

Thresholds can be expressed as a count (for example, at least 20 rows) or as a proportion (for example, at least 1% of training observations). The threshold is a modeling choice and should be assessed with cross-validation rather than selected solely for convenience.

Python example 11.9 — OneHotEncoder with built-in infrequent-category handling

The “Other” category

An "Other" group increases support by pooling categories, but it is semantically heterogeneous. Its effect should not be interpreted as a coherent real-world group. The category is a modeling device that trades detailed identity for statistical stability.

Risks of rare-category overfitting

• A category appearing once can perfectly coincide with one target value by chance.
• A tree may isolate a tiny category and create an unstable leaf.
• Target encoding can produce extreme estimates for tiny groups without smoothing.
• One-hot coefficients for rare levels can have high variance.
• A category may disappear or change spelling in production.

11.5 High-cardinality categorical features

High-cardinality features contain hundreds, thousands, or even millions of distinct values. Examples include product IDs, merchant IDs, postal codes, URLs, device models, job titles, and user-generated tags. Ordinary one-hot encoding may be impractical or statistically weak.

Problems with thousands of categories

• Large sparse matrices and high memory usage.
• Slow training and hyperparameter search.
• Rare-level overfitting.
• Unstable categories between training and production.
• Large model artifacts and difficult feature inspection.
• Potential identity leakage when the feature is actually an identifier.

Frequency encoding

Frequency encoding replaces each category with its count or proportion in the training data. It is compact and target-independent, but different categories with the same frequency receive the same value. The encoding expresses prevalence, not category identity.

Python example 11.10 — Frequency encoding

Target encoding

Target encoding replaces a category with a statistic of the target, commonly the mean target value for that category. For binary classification, this can approximate the historical positive-class rate. It is compact and powerful, especially for high-cardinality variables, but it directly uses labels and therefore has a serious leakage risk.

Python example 11.11 — Naive target encoding (illustrates the risky pattern)

Leakage risks in target encoding

If a row contributes to the statistic used to encode itself, the encoded feature contains information about its own target. The problem is particularly severe for rare categories: a category appearing once receives exactly that row’s target. Validation scores can become unrealistically high.

Cross-validated encoding

For training rows, out-of-fold target encoding computes each row’s encoded value using only other folds. For validation or test rows, the encoder is fitted on the available training data and applied to unseen rows. Smoothing is often added so that small groups shrink toward the global mean.

Python example 11.12 — Simple out-of-fold target encoding

Hashing

Feature hashing maps categories into a fixed number of columns using a hash function. The dimensionality is controlled regardless of the number of unique values, and no category vocabulary needs to be stored. The trade-off is collisions: different categories may map to the same column, and direct interpretability is reduced.

Python example 11.13 — Feature hashing

11.6 Unknown categories at prediction time

An unknown category is a label observed during transformation that was absent when the encoder was fitted. This is normal in real systems: new cities, products, merchants, devices, campaigns, or job titles appear continuously. Production-safe preprocessing treats unknown categories as an expected condition rather than an exceptional failure.

Configuring encoders safely

Python example 11.14 — Safe one-hot handling of unknown categories

Python example 11.15 — Safe ordinal handling with an explicit unknown code

Ensuring production robustness

• Fit encoders only on training data and persist the fitted preprocessing pipeline.
• Define behavior for unknown and missing categories explicitly.
• Validate input schemas before prediction.
• Log the frequency of unseen categories in production.
• Monitor category drift and the growth of the "Other" group.
• Use stable category definitions or controlled vocabularies when the domain allows them.
• Include unknown-category test cases in automated tests.

Python example 11.16 — Test production behavior explicitly

Encoding decision guide

Practical lab — Build separate numerical and categorical preprocessing procedures

Scenario

A subscription company wants to predict whether a customer will renew. The dataset contains numerical features, nominal categories, an ordered satisfaction variable, and a high-cardinality city field. Students must construct a leakage-safe preprocessing pipeline and verify that unseen categories can be transformed successfully.

Lab objectives

• Identify numerical, nominal, and ordinal features.
• Impute missing values separately by feature type.
• Scale numerical features.
• One-hot encode nominal categories with safe unknown handling.
• Ordinally encode the satisfaction level with a declared order.
• Group infrequent city labels automatically.
• Combine all transformations in ColumnTransformer.
• Fit preprocessing only on the training split.
• Inspect generated feature names and transformed dimensions.
• Verify an unseen production row can be processed.

Step 1 — Create the example dataset

Lab code 11.1 — Dataset

Step 2 — Separate X and y, then split

Lab code 11.2 — Train/test split before fitting encoders

Step 3 — Define feature groups

Lab code 11.3 — Feature groups

Step 4 — Build separate pipelines

Lab code 11.4 — Numerical, nominal, and ordinal pipelines

Step 5 — Combine transformations

Lab code 11.5 — ColumnTransformer

Step 6 — Inspect generated feature names

Lab code 11.6 — Feature names

Step 7 — Add the model to the pipeline

Lab code 11.7 — End-to-end classification pipeline

Step 8 — Test an unseen production category

Lab code 11.8 — Robustness test

Student deliverable

• A notebook containing the complete preprocessing and model pipeline.
• A table classifying each input feature as numerical, nominal, ordinal, or high-cardinality.
• An explanation of why each categorical encoding method was selected.
• The transformed training and test dimensions.
• A list of generated output feature names.
• A demonstration that unseen categories do not crash preprocessing.
• A short note explaining where leakage could occur if target encoding were introduced.

Evidence-based questions

1. Why is satisfaction_level encoded ordinally rather than one-hot encoded in this lab?
2. Why is city allowed to use an infrequent-category mechanism?
3. What happens to an unseen region when handle_unknown is configured safely?
4. Why is preprocessor.fit_transform used only on X_train while X_test uses transform?
5. Would target encoding city be automatically better? What validation would be required before choosing it?

Common mistakes and corrections

Chapter summary

• Categorical encoding converts qualitative labels into numerical representations while preserving useful semantics.
• Ordinal encoding is appropriate only when the categories have a genuine order.
• One-hot encoding is a strong default for low- and moderate-cardinality nominal variables.
• Rare-category grouping can improve stability and control dimensionality.
• High-cardinality features may require frequency encoding, target encoding, or hashing.
• Target encoding must be cross-validated or otherwise fitted without allowing each row to reveal its own target.
• Unknown-category behavior must be deliberately configured for robust production systems.
• ColumnTransformer and Pipeline provide a clean way to apply different preprocessing strategies to different feature groups without leakage.

Knowledge check

1. A feature contains Red, Green, and Blue with no natural ranking. Which encoding is the safest default?

Answer: One-hot encoding.

2. Why is coding Red=0, Green=1, Blue=2 risky?

Answer: It creates an artificial order and numerical distance.

3. What is the main benefit of sparse one-hot output?

Answer: It stores mostly-zero matrices efficiently.

4. What is the central danger of naive target encoding?

Answer: Target leakage and severe overfitting, especially for rare categories.

5. How should a fitted encoder behave when a new category appears in production?

Answer: Use an explicit safe strategy such as ignore, infrequent grouping, or an unknown code, depending on the encoder.

6. Should rare-category thresholds be determined from the test set?

Answer: No. They must be learned from training data only.