Chapter 1 — Introduction to Machine Learning
Level: Beginner to Intermediate
Recommended tools: Python • pandas • scikit-learn • Jupyter Notebook
Chapter Overview
Machine learning is one of the main technologies used to build systems that recognize patterns, estimate unknown quantities, classify observations, detect anomalies, and support decisions. This chapter introduces the vocabulary and conceptual foundations required for the rest of the course. It places supervised learning within the broader artificial intelligence landscape and explains how data, examples, algorithms, models, predictions, and errors interact in a machine learning project.
The chapter deliberately combines conceptual explanations with simple Python demonstrations. The code examples are not intended to provide a complete modeling workflow yet. Their purpose is to make the core ideas concrete before later chapters introduce data cleaning, preprocessing pipelines, model validation, metric selection, and hyperparameter optimization.
Learning Objectives
✓ Define machine learning and explain what it means for a system to learn from data.
✓ Differentiate artificial intelligence, machine learning, and deep learning.
✓ Compare traditional rule-based programming with a data-driven learning approach.
✓ Explain how historical data can be transformed into predictions, decisions, and recommendations.
✓ Describe supervised, unsupervised, semi-supervised, self-supervised, and reinforcement learning.
✓ Recognize common classification, regression, clustering, and non-machine-learning problems.
✓ Identify observations, features, targets, labels, models, algorithms, predictions, and losses in a dataset.
✓ Explain why the quality, representativeness, and governance of data affect model performance.
✓ Position supervised learning within the wider machine learning ecosystem.
FOUNDATION A central principle A machine learning model does not “understand” a problem in the human sense. It detects statistical regularities in the data it receives. Its usefulness therefore depends on the quality of the problem definition, the relevance of the data, and the appropriateness of the evaluation process. |
Chapter Roadmap
Part | Topic | Purpose |
|---|---|---|
| 1.1 | What is machine learning? | Definition, AI–ML–DL relationship, data-driven learning, predictions and decisions. |
| 1.2 | Main categories of machine learning | Supervised, unsupervised, semi-supervised, self-supervised, and reinforcement learning. |
| 1.3 | Examples of applications | Ten application areas and the structure of their prediction problems. |
| 1.4 | Components of a machine learning problem | Samples, features, targets, labels, algorithms, models, predictions, and losses. |
| Activity | Problem-type identification | Classification, regression, clustering, and alternative analytical approaches. |
1.1 What Is Machine Learning?
1.1.1 Definition of Machine Learning
Machine learning is a field of artificial intelligence concerned with methods that improve their performance on a task by learning from data or experience. Instead of requiring a developer to explicitly write every decision rule, a learning algorithm analyzes examples and estimates a model that can be applied to new, previously unseen cases.
A useful operational definition is the following:
DEFINITION Operational definition Machine learning is the process of using data and an algorithm to estimate a model that maps inputs to useful outputs, while aiming to perform well on new observations rather than only memorizing the training examples. |
This definition contains several essential ideas:
• There is a task to perform, such as predicting a category, estimating a value, grouping similar observations, or selecting an action.
• There is experience, usually represented by a dataset of examples or interactions.
• There is a measurable performance criterion, such as accuracy, prediction error, reward, or business cost.
• The system must generalize: it should remain useful when it receives data it did not see during training.
Learning versus memorization
A model that simply stores all training examples may appear successful when evaluated on those same examples, but it may fail on new data. Machine learning therefore emphasizes generalization. Generalization is the ability to transfer patterns learned from historical data to future or unseen observations drawn from a sufficiently similar environment.
The distinction between learning and memorization becomes especially important when a model is very complex, the dataset is small, the data contains duplicated records, or evaluation is performed incorrectly. Later chapters will examine overfitting, cross-validation, and test sets in detail.
Machine learning | A computational approach in which patterns are estimated from data so that a system can make predictions, decisions, or representations on new inputs. |
Generalization | The ability of a trained model to perform well on observations that were not used to fit the model. |
Training | The process of adjusting a model using data and an objective function. |
Inference | The use of a trained model to generate predictions or outputs for new inputs. |
1.1.2 Artificial Intelligence, Machine Learning, and Deep Learning
Artificial intelligence, machine learning, and deep learning are related concepts, but they are not interchangeable. Artificial intelligence is the broadest field. Machine learning is one family of approaches within artificial intelligence. Deep learning is a specialized family of machine learning methods based on multilayer neural networks.
ARTIFICIAL INTELLIGENCE Systems designed to perform tasks associated with intelligent behavior. |
MACHINE LEARNING AI approaches that learn patterns from data or experience. |
DEEP LEARNING Machine learning based on deep neural-network architectures. |
Artificial intelligence
Artificial intelligence is the broad discipline of designing machines or software systems capable of tasks that are commonly associated with intelligent behavior. These tasks may include reasoning, planning, perception, language processing, problem solving, decision making, and autonomous action.
An AI system does not necessarily use machine learning. A rule-based expert system, a search algorithm, a constraint solver, or a planning program can be considered artificial intelligence even when it does not learn from data.
Machine learning
Machine learning is used when useful rules are difficult to specify manually but examples, measurements, or interaction data are available. The algorithm identifies statistical relationships and expresses them through a model. Classical machine learning includes linear models, decision trees, support vector machines, nearest-neighbor methods, probabilistic models, and ensemble methods.
Deep learning
Deep learning uses neural networks containing multiple processing layers. These networks can automatically learn increasingly abstract representations from raw or lightly processed data. Deep learning is particularly influential in computer vision, speech recognition, natural language processing, generative AI, and other domains with large, complex, and high-dimensional datasets.
Deep learning is not automatically the best choice for every problem. For many structured tabular datasets, classical machine learning methods can be faster, easier to interpret, less data-hungry, and competitive in performance.
Concept | Scope | Typical methods | Typical strengths |
|---|---|---|---|
| Artificial intelligence | Broadest area | Rules, search, planning, optimization, ML | Can combine logic, knowledge, and learning. |
| Machine learning | Subset of AI | Linear models, trees, SVM, ensembles, neural networks | Learns predictive patterns from examples. |
| Deep learning | Subset of ML | CNNs, RNNs, Transformers, deep networks | Learns complex representations from large-scale data. |
CAUTION Do not confuse the hierarchy All deep learning is machine learning, and all machine learning is part of artificial intelligence. However, not all machine learning is deep learning, and not all artificial intelligence uses machine learning. |
1.1.3 Traditional Programming versus Machine Learning
Traditional programming and machine learning differ mainly in how the relationship between inputs and outputs is constructed. In traditional programming, a developer writes explicit rules. In supervised machine learning, the developer provides examples of inputs and expected outputs, and a learning algorithm estimates rules in the form of a trained model.
Approach | Inputs | Processing | Output |
|---|---|---|---|
| Traditional programming | Data + explicitly written rules | The program executes the rules | Answers or actions |
| Supervised machine learning training | Data + known answers | A learning algorithm estimates patterns | A trained model |
| Machine learning inference | New data + trained model | The model applies learned relationships | Predictions or scores |
Example: manually programmed spam rules
A traditional spam filter might search for manually selected words or patterns. The programmer decides the rules and thresholds. Such a program can be understandable and effective for simple cases, but it may require constant manual updates as spam tactics change.
Python Example 1.1 — A rule-based decision def rule_based_spam_filter(subject, body): |
The advantages of this approach are transparency and direct control. Its limitations are brittleness, incomplete coverage, and the cost of manually maintaining a growing collection of rules.
Example: learning a spam model from labeled messages
A machine learning approach starts with messages labeled as spam or legitimate. A feature extraction process converts the text into numerical values. A learning algorithm then estimates which patterns are associated with each class. The resulting model can combine thousands of weak signals rather than relying on a small number of manually written rules.
Python Example 1.2 — Learning a simple spam classifier from sklearn.feature_extraction.text import TfidfVectorizer |
LIMITATION Important limitation This miniature dataset is only a teaching example. A real spam detector requires much more data, careful validation, privacy controls, language coverage, protection against data leakage, and continuous monitoring. |
When should rules be preferred?
Machine learning is not a replacement for all traditional software logic. Explicit rules are often preferable when requirements are stable, exact behavior is required, errors are unacceptable, examples are unavailable, or the decision can be expressed clearly and completely. Many production systems combine rules and machine learning: rules enforce hard constraints, while models estimate uncertain quantities.
1.1.4 Learning Patterns from Historical Data
Historical data records past observations, outcomes, interactions, or measurements. In supervised learning, each training example typically contains inputs and a known target. The algorithm searches for relationships that reduce prediction error on the training examples while being controlled to support generalization.
Suppose a dataset contains information about houses and their sale prices. The features may include surface area, location, number of rooms, age, and condition. The target is the sale price. A regression algorithm analyzes how the target changes with the features and produces a model that estimates prices for houses not present in the training data.
Patterns can be simple or complex
• A linear pattern: price tends to increase as surface area increases.
• A threshold pattern: failure risk rises sharply above a temperature limit.
• An interaction: the effect of surface area may depend on location.
• A temporal pattern: demand rises during certain months or hours.
• A high-dimensional pattern: an image class depends on the arrangement of many pixels.
Correlation is not causation
Machine learning models often exploit associations rather than causal relationships. A feature can improve prediction without causing the target. For example, an umbrella may predict rain because umbrellas are used when rain is expected; the umbrella does not cause the rain. This distinction matters when a model is used to recommend interventions rather than merely predict outcomes.
CONCEPT CHECK Prediction versus explanation A highly predictive model is not automatically a causal explanation. Predictive modeling asks, “What output is likely?” Causal analysis asks, “What would change if we intervened?” These questions require different assumptions and methods. |
Historical data must represent future use
A model can only learn from the information present in its data. If future cases differ significantly from the training examples, performance can deteriorate. This may happen because customer behavior changes, sensors are replaced, policies change, new categories appear, or the data collection process is modified. The relationship between training conditions and deployment conditions must therefore be examined explicitly.
1.1.5 Predictions, Decisions, and Recommendations
A trained model usually produces an intermediate output rather than a complete operational decision. Understanding the distinction between prediction, decision, and recommendation prevents a common design error: treating a model score as if it were automatically the correct action.
Prediction | An estimated class, value, probability, score, or future outcome produced by a model for a given input. |
Decision | An action selected using predictions together with rules, costs, constraints, policies, and human judgment. |
Recommendation | A ranked or suggested option intended to help a user or system choose an action. |
System stage | Example in credit assessment | Example in maintenance |
|---|---|---|
| Prediction | Estimated probability of repayment failure | Estimated probability of equipment failure |
| Decision rule | Refer applications above a risk threshold for review | Schedule inspection when risk and downtime cost justify it |
| Final action | Approve, reject, or request additional information | Continue operation, inspect, repair, or replace |
Thresholds connect probabilities to actions
A binary classifier may output a probability such as 0.72. The system still needs a threshold or decision policy. A low threshold can detect more positive cases but may create more false alarms. A high threshold can reduce false alarms but miss more true cases. The threshold should therefore reflect costs, risk tolerance, capacity, and the intended use of the model.
Python Example 1.3 — Converting a prediction into a recommendation def maintenance_action(failure_probability, threshold=0.65): |
Human-in-the-loop systems
In sensitive domains, the model should support rather than replace qualified human judgment. A human-in-the-loop design may route uncertain or high-risk cases to an expert, provide explanations, allow correction of incorrect information, and preserve accountability. The degree of automation should depend on the consequences of errors and the legal and ethical context.
1.1.6 Role of Data in Machine Learning Systems
Data is the evidence from which a machine learning model estimates patterns. Algorithms are important, but a sophisticated algorithm cannot compensate for data that is irrelevant, incorrectly labeled, systematically biased, or inconsistent with deployment conditions. The quality of a learning system is therefore inseparable from the quality and governance of its data.
Data quality dimensions
Dimension | Guiding question |
|---|---|
| Accuracy | Values correctly describe the real-world objects or events. |
| Completeness | Necessary variables and labels are sufficiently available. |
| Consistency | Units, formats, categories, and definitions are applied uniformly. |
| Timeliness | Data reflects the period and conditions in which the model will be used. |
| Representativeness | Important groups and operating conditions are adequately covered. |
| Label quality | Targets are reliable, meaningful, and produced with a clear procedure. |
| Traceability | The origin, processing steps, and versions of the data are documented. |
| Legality and ethics | Collection and use comply with consent, privacy, fairness, and applicable rules. |
More data is not always better data
Increasing dataset size can improve model stability, but only when the added data is relevant and trustworthy. A large dataset containing duplicated records, incorrect labels, obsolete behavior, or sampling bias may produce a confidently wrong model. Data quality, coverage, and alignment with the prediction objective are often more important than raw volume.
Training, validation, and test data
Machine learning development usually separates data into distinct roles. Training data is used to fit the model. Validation data or cross-validation is used to compare alternatives and tune settings. Test data is reserved for a final, relatively unbiased estimate of performance. The same information must not leak between these roles, because leakage can create results that appear excellent but do not generalize.
A first look at features and a target in Python
Python Example 1.4 — Separating features and target import pandas as pd |
GOOD PRACTICE Data is contextual A column is not automatically a good feature merely because it exists. A useful feature must be relevant to the target, available at prediction time, legally and ethically usable, and sufficiently stable under deployment conditions. |
1.2 Main Categories of Machine Learning
Machine learning paradigms differ in the type of feedback available during learning. The most important distinction is whether the system receives explicit target labels, only raw observations, partially labeled data, automatically constructed learning signals, or rewards generated through interaction.
1.2.1 Supervised Learning
Supervised learning uses examples in which the desired output is known. Each training observation contains input features and a target. The learning algorithm estimates a mapping from inputs to targets and is evaluated on its ability to predict targets for new observations.
The two principal supervised tasks are classification and regression:
• Classification predicts a discrete category, such as spam/not spam, approved/rejected, or one of several image classes.
• Regression predicts a numerical quantity, such as price, temperature, demand, or remaining useful life.
Supervised learning is the central focus of this course. Later chapters will cover data preparation, candidate algorithms, evaluation metrics, cross-validation, model interpretation, and deployment.
Typical supervised workflow
1. Collect observations for which the target is known.
2. Represent each observation using appropriate features.
3. Separate data for training and evaluation.
4. Fit a model by minimizing a loss or maximizing an objective.
5. Evaluate the trained model on unseen observations.
6. Use the model to generate predictions for new inputs.
1.2.2 Unsupervised Learning
Unsupervised learning works with data that does not contain an explicit target label for the task. The system searches for structure, similarity, lower-dimensional representations, or unusual observations. The absence of labels does not mean the task has no objective; it means that the desired answer is not directly supplied for every example.
Common unsupervised tasks include:
• Clustering: grouping similar observations.
• Dimensionality reduction: representing high-dimensional data with fewer variables.
• Anomaly detection: identifying observations that differ substantially from typical patterns.
• Association analysis: discovering items or events that frequently occur together.
Unsupervised results often require domain interpretation. A clustering algorithm may produce groups, but a human or downstream analysis must determine whether the groups are meaningful and useful.
1.2.3 Semi-Supervised Learning
Semi-supervised learning combines a relatively small labeled dataset with a larger unlabeled dataset. It is useful when collecting raw data is easy but assigning reliable labels is expensive, slow, or dependent on experts. Medical images, industrial inspections, audio recordings, and web documents are common examples.
The central assumption is that the structure of the unlabeled data can help the model learn a better representation or decision boundary than the labeled examples alone. Techniques include pseudo-labeling, consistency regularization, graph-based methods, and hybrid supervised-unsupervised objectives.
CAUTION Main risk Incorrect pseudo-labels can reinforce model errors. Semi-supervised systems require careful confidence controls, validation, and analysis of whether labeled and unlabeled data come from compatible distributions. |
1.2.4 Self-Supervised Learning
Self-supervised learning creates training signals from the internal structure of unlabeled data. Instead of relying on human-provided labels, the system solves a pretext task constructed automatically. Examples include predicting masked words, predicting missing image regions, comparing different augmented views of the same observation, or predicting future segments of a signal.
The main objective is usually to learn a useful representation. The pretrained representation can then be adapted to a downstream task using a smaller labeled dataset. Modern language models and many vision models rely heavily on self-supervised pretraining.
Self-supervised learning differs from unsupervised learning mainly in the explicit construction of a supervised-like learning signal from the data itself. It differs from conventional supervised learning because the targets are generated automatically rather than manually annotated for the final task.
1.2.5 Reinforcement Learning
Reinforcement learning studies an agent that interacts with an environment. At each step, the agent observes a state, selects an action, receives a reward, and transitions to another state. The objective is to learn a policy that maximizes long-term cumulative reward rather than predict a fixed label supplied for each observation.
Agent | The decision-making system that selects actions. |
Environment | The external system or simulation with which the agent interacts. |
State | Information describing the current situation available to the agent. |
Action | A choice the agent can make. |
Reward | A numerical feedback signal indicating the immediate value of an action or outcome. |
Policy | A strategy that maps states or observations to actions. |
Reinforcement learning is appropriate when actions influence future situations and delayed consequences matter. Examples include robot control, game playing, resource allocation, and adaptive recommendation strategies. It is generally more difficult to develop safely than standard supervised learning because exploration can be costly and the reward function may not fully capture the desired behavior.
1.2.6 Comparison of Learning Paradigms
Paradigm | Learning signal | Primary goal | Typical task | Main challenge |
|---|---|---|---|---|
| Supervised | Inputs with explicit targets | Predict target for new inputs | Classification, regression | Requires reliable labels |
| Unsupervised | Inputs without target labels | Discover structure or representations | Clustering, anomaly detection | Results may be hard to validate |
| Semi-supervised | Small labeled set + large unlabeled set | Improve predictive learning with unlabeled data | Image or document classification | Error propagation from pseudo-labels |
| Self-supervised | Unlabeled data with automatically generated targets | Learn reusable representations | Masked-token or contrastive learning | Pretext task may not align with final task |
| Reinforcement | Interactions, states, actions, rewards | Learn a policy maximizing cumulative reward | Control and sequential decisions | Exploration, safety, delayed feedback |
Choosing a paradigm
The choice of paradigm begins with the available information and the intended output. When historical examples include the answer to be predicted, supervised learning is usually the natural starting point. When no target exists and the goal is to explore structure, unsupervised learning may be more appropriate. When labels are scarce but unlabeled data is abundant, semi-supervised or self-supervised approaches may help. When decisions alter future states and rewards are delayed, reinforcement learning may be required.
Python Example 1.5 — A conceptual paradigm-selection helper def suggest_learning_paradigm( |
1.3 Examples of Machine Learning Applications
The following examples illustrate how application descriptions can be translated into machine learning problems. For each case, it is necessary to identify the unit of observation, the available inputs, the target, the prediction moment, and the cost of errors. The same application domain can sometimes support several different tasks.
Application | Observation | Possible features | Target | Task | Key consideration |
|---|---|---|---|---|---|
| Email spam detection | One incoming email | Sender metadata, text, links, attachments, historical behavior | Spam or legitimate | Binary classification | False positives may hide legitimate messages; false negatives expose users to spam or fraud. |
| Fraud detection | One transaction or account event | Amount, time, merchant, device, location, behavior history | Fraudulent or legitimate; sometimes a risk score | Classification or anomaly detection | Fraud is rare, labels may be delayed, and attackers adapt. |
| Medical diagnosis | One patient encounter, image, or examination | Symptoms, measurements, medical images, history | Disease category or probability | Classification | High stakes, data privacy, subgroup performance, and human oversight are essential. |
| Customer churn prediction | One customer at a defined date | Usage, payments, complaints, tenure, engagement | Will churn within a future window | Binary classification | The target window and intervention timing must be clearly defined. |
| House price estimation | One property transaction | Location, size, rooms, age, condition, market context | Sale price | Regression | Market changes and location effects can reduce stability. |
| Credit risk assessment | One application or borrower | Income, debt, payment history, requested amount | Default probability or risk class | Classification or probability estimation | Fairness, explainability, regulation, and data quality are critical. |
| Equipment failure prediction | One machine or operating window | Vibration, temperature, pressure, alarms, maintenance history | Failure within a horizon or remaining useful life | Classification or regression | Rare failures, sensor drift, and maintenance-induced label ambiguity. |
| Image classification | One image or region | Pixel values or learned visual features | Object or scene class | Multiclass or multilabel classification | Dataset coverage, annotation quality, and distribution shift. |
| Demand forecasting | One product-location-time interval | Historical sales, calendar, promotions, prices, weather | Future demand quantity | Regression or time-series forecasting | Temporal validation and changing behavior are central. |
| Student performance prediction | One student-course-period record | Prior results, attendance, activity, assessments | Score, pass/fail, or support need | Regression or classification | Models must support students without stigmatizing or creating self-fulfilling outcomes. |
1.3.1 Email Spam Detection
Spam detection is commonly formulated as binary classification. Each email is an observation, and the target indicates whether the message is spam or legitimate. Features may come from the subject, body, sender domain, number of links, language patterns, attachment types, and prior user interactions.
A production spam filter often combines machine learning with security rules, reputation systems, malware scanning, and user feedback. Evaluation must consider both false positives and false negatives. A false positive can prevent an important message from reaching the inbox, while a false negative can expose a user to fraud or malicious content.
1.3.2 Fraud Detection
Fraud detection may be supervised when reliable historical fraud labels exist. However, confirmed fraud is typically rare and may only be known after an investigation. Unsupervised anomaly detection can complement classification by identifying unusual behavior that does not match known fraud patterns.
The model output is usually a risk score rather than a final legal conclusion. The score may trigger additional authentication, manual review, delayed processing, or transaction blocking according to policy and risk level.
1.3.3 Medical Diagnosis
Medical machine learning can classify images, estimate disease risk, predict complications, or prioritize cases. The model must be evaluated in the population and clinical workflow in which it will be used. Dataset size alone is insufficient; label quality, device differences, hospital practices, subgroup representation, and prospective validation are major concerns.
RESPONSIBLE USE High-stakes use A medical model should not be treated as a substitute for professional diagnosis without appropriate clinical validation, governance, and regulatory approval. The appropriate role may be triage, decision support, quality control, or research rather than autonomous diagnosis. |
1.3.4 Customer Churn Prediction
Churn prediction estimates whether a customer is likely to stop using a service during a specified future period. The target must include a clear time horizon, such as “churn within the next 30 days.” A vague target such as “customer will leave” is not operationally sufficient.
The model is useful only when the prediction arrives early enough for an appropriate intervention. A system that identifies churn after the customer has already cancelled is an example of target leakage or incorrect prediction timing.
1.3.5 House Price Estimation
House price estimation is a regression problem because the target is numerical. The value of a property may depend on location, size, age, condition, local services, market trends, and many interactions. The evaluation metric should reflect the practical cost of errors. Mean absolute error is easy to interpret in currency units, while squared-error metrics penalize large mistakes more strongly.
1.3.6 Credit Risk Assessment
Credit risk models estimate the probability that a borrower will fail to meet repayment obligations. The target must specify what counts as default and the observation period. A model may be statistically accurate while still producing unfair or legally problematic outcomes if historical data reflects unequal access, biased decisions, or proxy variables for protected characteristics.
1.3.7 Equipment Failure Prediction
Predictive maintenance systems analyze sensor measurements and operational history to estimate the probability of failure or remaining useful life. The unit of observation might be one machine-day, one operating cycle, or a fixed sensor window. The selected representation must match the time at which maintenance decisions are made.
Failure labels are often difficult: parts may be replaced preventively before failing, sensor systems may be incomplete, and maintenance actions change the future outcome. These issues require careful target definition.
1.3.8 Image Classification
Image classification assigns one or more labels to an image. Classical machine learning may use engineered visual features, while deep learning commonly learns features directly from pixels. The dataset must represent variations in lighting, viewpoint, background, camera quality, object appearance, and other factors expected in deployment.
1.3.9 Demand Forecasting
Demand forecasting estimates future quantities for products, services, energy, traffic, or resources. Although it predicts a numerical target and therefore resembles regression, the temporal structure requires specialized validation. Randomly mixing future observations into the training set can cause leakage. Models should generally be trained on earlier periods and evaluated on later periods.
1.3.10 Student Performance Prediction
Student performance models can estimate scores, pass/fail outcomes, dropout risk, or the need for support. The educational purpose should be explicit. The model should be used to allocate assistance and identify barriers rather than label students as permanently weak. Predictions can become self-fulfilling when they influence expectations and opportunities.
PURPOSE A useful design question Before training a model, ask: “What beneficial action becomes possible because this prediction exists?” If no clear, ethical, and feasible action follows, the prediction may have limited value. |
1.4 Components of a Machine Learning Problem
A supervised machine learning problem can be represented using a set of observations, a collection of input variables, a target, an algorithm, and an evaluation objective. Precise terminology is important because several words are related but not identical.
1.4.1 Observations or Samples
An observation, sample, instance, record, or example is one unit described by the dataset. In a table, observations are usually represented by rows. The correct unit depends on the decision problem. It might be a customer, transaction, patient visit, image, machine cycle, or time window.
The unit of observation must be defined before creating the dataset. Mixing different units in the same table can create ambiguous targets and invalid evaluation. Multiple records from the same entity also require special splitting strategies to prevent information leakage.
1.4.2 Input Variables and Features
An input variable is a piece of information supplied to the model. A feature is a numerical or encoded representation used by the learning algorithm. In simple tabular problems, a column may directly serve as a feature. In other cases, feature extraction transforms raw data into a useful representation.
Examples include:
• Raw variable: date of transaction; engineered features: month, day of week, holiday indicator, and time since last purchase.
• Raw variable: email text; features: word frequencies, embeddings, link count, and message length.
• Raw variable: vibration signal; features: frequency-band energy, peak amplitude, statistical descriptors, or learned neural representations.
• Raw variable: image; features: handcrafted descriptors or representations learned by a convolutional neural network.
1.4.3 Target Variable and Labels
The target variable is the outcome the model is trained to predict. In supervised learning, the known target values in the training data are often called labels. The word “label” is most commonly used for classification, but it can also refer broadly to supervised targets.
A high-quality target should be clearly defined, consistently measured, relevant to the intended action, and available for enough historical examples. It must also be unavailable as an input at prediction time; otherwise the model may simply copy or indirectly reconstruct the answer.
1.4.4 Training Examples
A training example combines an input representation with its known target. For tabular supervised learning, one example can be written as a pair (xᵢ, yᵢ), where xᵢ is the feature vector for observation i and yᵢ is the corresponding target. A training dataset contains many such pairs.
The examples should be independent enough for the evaluation design and representative enough of the cases the model will later encounter. Duplicated or near-duplicated examples can produce inflated performance when copies appear in both training and test data.
1.4.5 Model
A model is the mathematical or computational function produced or configured by the learning process. It maps input features to an output such as a class, numerical estimate, probability, score, or representation. Examples include a fitted linear equation, a trained decision tree, a collection of trees, or a neural network with learned weights.
The word “model” can refer to both the model family and the fitted instance. Logistic regression is a model family; a logistic regression fitted to a particular training dataset with specific coefficients is a trained model.
1.4.6 Learning Algorithm
A learning algorithm is the procedure used to estimate the model from data. It defines how candidate model parameters are evaluated and updated. For example, a linear regression algorithm may choose coefficients that minimize squared prediction error. A decision-tree algorithm searches for feature splits that improve node purity or reduce target variance.
Algorithm and model are therefore related but different: the algorithm is the procedure, while the fitted model is the result of applying the procedure to data.
1.4.7 Prediction
A prediction is the output generated by the trained model for a particular input. Depending on the task, it may be a class label, a numerical value, a probability distribution, a risk score, or a ranked list. The notation ŷ is commonly used for a predicted target, while y represents the observed target.
Predictions should be accompanied by enough context for their use. A score without a definition, time horizon, threshold, or confidence interpretation can easily be misunderstood.
1.4.8 Error or Loss
Error describes the difference between a prediction and the true target. A loss function converts this difference into a numerical quantity used during training or evaluation. The learning algorithm seeks model parameters that reduce an objective based on the loss.
Different losses emphasize different types of mistakes. Squared error gives large errors greater influence. Absolute error treats the magnitude of error linearly. Classification losses such as log loss penalize incorrect probability estimates, especially confident incorrect predictions.
Term | Meaning | House-price example | Spam example |
|---|---|---|---|
| Observation/sample | One unit in the dataset | One sold house | One email |
| Input variable | Available descriptive information | Surface area, location, age | Text, sender, links |
| Feature | Numerical representation used by the model | Scaled area, encoded district | TF–IDF word values |
| Target/label | Known output to predict | Sale price | Spam or legitimate |
| Training example | Features paired with a known target | House characteristics + price | Email representation + label |
| Algorithm | Procedure that learns from examples | Linear regression fitting | Logistic regression fitting |
| Model | Fitted mapping from features to outputs | Estimated pricing equation | Trained spam classifier |
| Prediction | Model output for new input | Estimated price | Spam probability |
| Loss/error | Numerical measure of incorrectness | Difference from actual price | Penalty for wrong class probability |
A minimal supervised learning example
The following example uses the Iris dataset included with scikit-learn. Each observation is a flower. The features are measurements of sepals and petals. The target is the species. Because the target contains three categories, the task is multiclass classification.
Python Example 1.6 — Observations, features, labels, model, prediction, and error from sklearn.datasets import load_iris |
This example already contains several ideas that will be studied carefully later: a train-test split, stratification, model fitting, prediction, and evaluation. At this stage, the most important objective is to identify the role of each object rather than optimize the model.
Practical Activity — Identify the Appropriate Analytical Approach
In this activity, students analyze real-world scenarios and decide whether the main task is classification, regression, clustering, or another analytical approach. The objective is not merely to recognize keywords; students must identify the expected output and determine whether labeled examples are available.
Activity instructions
1. Read each scenario and identify the unit of observation.
2. State the intended output in one precise sentence.
3. Determine whether the output is categorical, numerical, a group assignment, or something else.
4. Select classification, regression, clustering, or another approach.
5. Justify the choice and list one possible evaluation criterion.
6. Identify one risk related to data quality, leakage, fairness, or deployment.
Scenario | Situation |
|---|---|
| A | A university wants to predict whether an admitted student will enroll before the registration deadline. |
| B | A property platform wants to estimate the selling price of a new apartment. |
| C | A retailer wants to discover groups of customers with similar purchasing behavior, without predefined customer types. |
| D | An engineer wants to calculate the average temperature recorded by each sensor during the last hour. |
| E | A hospital wants to assign a medical image to one of four diagnostic categories. |
| F | A factory wants to estimate the number of operating hours remaining before a component must be replaced. |
| G | A bank wants to identify unusual transactions for which confirmed fraud labels are unavailable. |
| H | A school wants to sort students alphabetically by family name. |
| I | A delivery company wants to choose the shortest route between a depot and several destinations using a road network. |
| J | A streaming service wants to predict the next movie category a user is likely to watch. |
| K | A researcher wants to summarize a dataset using its mean, median, standard deviation, and quartiles. |
| L | A call center wants to predict the number of minutes required to resolve a support request. |
Student worksheet
Scenario | Observation | Expected output | Approach | Metric/criterion | Risk or limitation |
|---|---|---|---|---|---|
| A | |||||
| B | |||||
| C | |||||
| D | |||||
| E | |||||
| F | |||||
| G | |||||
| H | |||||
| I | |||||
| J | |||||
| K | |||||
| L |
Optional Python starter
Python Activity 1.1 — Record and review your choices scenarios = { |
Suggested solutions and reasoning
Scenario | Suggested approach | Reasoning |
|---|---|---|
| A | Classification | The output is a categorical yes/no enrollment outcome. A metric could be recall, precision, F1-score, or calibration depending on the intended intervention. |
| B | Regression | The output is a continuous monetary value. MAE or RMSE could be used. |
| C | Clustering | No predefined segment labels are supplied; the objective is to discover groups based on similarity. |
| D | Other: descriptive statistics | The requested result is a direct aggregation, not a learned prediction. A simple mean calculation is sufficient. |
| E | Classification | The output is one of four diagnostic categories. Per-class recall and a confusion matrix would be important. |
| F | Regression | Remaining useful life is numerical. MAE, RMSE, or a domain-specific asymmetric error measure could be appropriate. |
| G | Unsupervised anomaly detection | Confirmed labels are unavailable; the initial objective is to flag unusual transactions. Human investigation may later provide labels. |
| H | Other: deterministic sorting | Alphabetical sorting is a fully specified algorithmic operation and does not require machine learning. |
| I | Other: graph optimization | Shortest-path algorithms solve the problem directly when the road network and costs are known. |
| J | Classification or recommendation | Predicting a category is classification; generating a ranked set of specific movies is a recommendation problem. |
| K | Other: descriptive statistics | The task summarizes observed data and does not predict unseen targets. |
| L | Regression | Resolution time is numerical. MAE may be particularly interpretable in minutes. |
Extension questions
• How would Scenario A change if the university wanted to estimate the probability of enrollment rather than only produce a yes/no label?
• Could Scenario G later become supervised learning? What additional information would be required?
• For Scenario J, what is the difference between predicting a category and ranking individual movies?
• Which scenarios require temporal splitting rather than a random train-test split?
• Which scenarios involve high-stakes decisions and therefore require human oversight or stronger governance?
Expected Outcome
After completing this chapter and activity, students should understand where supervised learning fits within the broader machine learning field. They should be able to distinguish a supervised prediction problem from clustering, descriptive statistics, deterministic programming, optimization, and sequential decision-making problems.
A successful student should be able to take a short application description and identify:
• the observation or unit being analyzed;
• the inputs available before the prediction is made;
• the target or output expected from the system;
• whether the output is categorical or numerical;
• whether historical labels are available;
• the likely machine learning paradigm;
• one appropriate evaluation criterion;
• one important risk or limitation.
Chapter Summary
• Machine learning estimates patterns from data so that a system can produce useful outputs for new observations.
• Generalization, rather than memorization of training examples, is the core objective of predictive modeling.
• Artificial intelligence is the broad field; machine learning is a subset of AI; deep learning is a subset of machine learning.
• Traditional programming relies on explicitly written rules, while supervised learning estimates a model from inputs paired with known outputs.
• Predictions are not automatically decisions. Policies, thresholds, costs, constraints, and human judgment determine actions.
• Data quality, representativeness, target definition, legality, and deployment alignment strongly influence model usefulness.
• Supervised learning uses labeled examples; unsupervised learning discovers structure; semi-supervised learning combines labeled and unlabeled data; self-supervised learning constructs signals from raw data; reinforcement learning learns through rewards and interaction.
• Classification predicts categories, regression predicts numerical values, and clustering groups observations without predefined class labels.
• A supervised problem contains observations, features, targets or labels, training examples, an algorithm, a fitted model, predictions, and an error or loss measure.
• Not every analytical problem requires machine learning. Direct calculations, deterministic rules, search, optimization, and descriptive statistics may be more appropriate.
Key Terminology
Term | Meaning |
|---|---|
| Artificial intelligence | The broad field concerned with systems that perform tasks associated with intelligent behavior. |
| Machine learning | Methods that improve task performance by learning from data or experience. |
| Deep learning | Machine learning based on multilayer neural networks. |
| Supervised learning | Learning from inputs paired with known targets. |
| Unsupervised learning | Learning structure from data without explicit target labels. |
| Classification | Prediction of a discrete category or set of categories. |
| Regression | Prediction of a continuous numerical value. |
| Clustering | Grouping observations according to similarity without predefined group labels. |
| Observation | One unit, row, case, or instance in a dataset. |
| Feature | A numerical or encoded input used by a learning algorithm. |
| Target | The outcome a supervised model is trained to predict. |
| Label | A known target value, especially a classification category. |
| Algorithm | A procedure used to estimate a model from data. |
| Model | A fitted function that maps inputs to outputs. |
| Prediction | The output generated by a model for an input. |
| Loss | A numerical measure of prediction error used for learning or evaluation. |
| Generalization | Performance on new observations rather than only the training examples. |
| Data leakage | Information enters training or evaluation in a way that creates unrealistically optimistic performance. |
Knowledge Check
1. Explain machine learning in your own words without using the phrase “artificial intelligence.”
2. Why is generalization more important than achieving perfect performance on the training data?
3. Give one example of an AI system that does not require machine learning.
4. Explain the difference between a learning algorithm and a trained model.
5. A model outputs a fraud probability of 0.80. Why is this not yet a complete decision?
6. What distinguishes classification from regression?
7. When might clustering be more appropriate than classification?
8. How does self-supervised learning obtain targets without manual annotation?
9. Why can historical data produce biased or unreliable predictions?
10. Name two problems that can be solved without machine learning and state the alternative method.
Short practical assignment
Select one application relevant to your studies or professional field. Write a one-page problem formulation containing the following elements:
• The real-world objective and intended user.
• The unit of observation.
• The inputs available at prediction time.
• The target variable and its time horizon.
• The most appropriate learning paradigm.
• A baseline that does not use machine learning.
• One technical evaluation metric.
• One practical, ethical, or operational risk.
Instructor Notes and Suggested Timing
Session | Content | Suggested duration | Teaching method |
|---|---|---|---|
| 1 | Definition; AI, ML, and deep learning | 60–75 min | Lecture, hierarchy exercise, examples |
| 2 | Traditional programming versus learning from data | 60 min | Code demonstration and discussion |
| 3 | Learning paradigms | 75–90 min | Comparative lecture and scenario classification |
| 4 | Application case studies | 75 min | Small-group analysis |
| 5 | Components of a supervised problem | 75 min | Dataset walkthrough and Python demo |
| 6 | Practical activity and correction | 60–90 min | Individual work, peer discussion, correction |
NEXT STEP Transition to Chapter 2 The next chapter should move from broad machine learning concepts to the precise structure of supervised learning problems, with deeper treatment of classification, regression, inputs, targets, and the conditions required for reliable labeled data. |