Chapter 39 — Building a Basic Prediction Application
Learning objectives
- Build a batch-prediction script that loads a persisted pipeline, validates new data, generates predictions, and exports results.
- Explain the request-response structure of a simple prediction REST API.
- Apply input validation before a model receives user-supplied values.
- Compare Streamlit, Flask, FastAPI, and Django as simple interfaces around a trained model.
- Identify production concerns including latency, concurrency, authentication, logging, privacy, and dependency management.
- Create either a batch prediction program or a basic interactive prediction interface using the persisted model from Chapter 38.
Chapter focus Chapter 38 produced a reusable model artifact. Chapter 39 turns that artifact into a usable prediction workflow. The key engineering principle is to keep model inference predictable: validate inputs, reuse the exact fitted pipeline, return explicit outputs, and handle failures without silently producing unreliable predictions. |
39.1 Batch prediction script
Batch prediction is often the simplest way to operationalize a model. A file containing new observations is read, validated against the expected schema, scored by the saved pipeline, and written to a result file. This pattern is appropriate when predictions do not need to be returned immediately to an interactive user.
| Stage | Purpose | Typical failure to detect |
|---|---|---|
| 1. Read new data | Load the incoming CSV or table into a DataFrame. | Missing file, encoding problem, malformed rows. |
| 2. Check schema | Confirm required columns, reject unknown or missing columns, and reorder features. | Wrong columns, unit changes, unexpected data types. |
| 3. Load artifact | Load the trusted persisted pipeline and associated metadata. | Missing artifact, incompatible environment, damaged file. |
| 4. Predict | Call the saved pipeline instead of manually repeating preprocessing. | Invalid values, transformation errors. |
| 5. Export | Write predictions and optional probabilities to a new file. | Wrong output path, accidental overwrite, incomplete results. |
Python example — load the trusted pipeline and metadata
| from pathlib import Path import json import joblib import pandas as pd ARTIFACT_DIR = Path("artifacts/model_v1") MODEL_PATH = ARTIFACT_DIR / "pipeline.joblib" METADATA_PATH = ARTIFACT_DIR / "metadata.json" pipeline = joblib.load(MODEL_PATH) # Load only trusted artifacts. with METADATA_PATH.open("r", encoding="utf-8") as file: metadata = json.load(file) expected_features = metadata["feature_list"] threshold = metadata.get("decision_threshold", 0.50) |
Python example — validate and order input columns
| def validate_columns(frame, expected_features): received = list(frame.columns) missing = [name for name in expected_features if name not in received] extra = [name for name in received if name not in expected_features] if missing: raise ValueError(f"Missing columns: {missing}") if extra: raise ValueError(f"Unexpected columns: {extra}") return frame[expected_features] new_data = pd.read_csv("new_data.csv") X_new = validate_columns(new_data, expected_features) |
Python example — score the batch and export a CSV
| probability = pipeline.predict_proba(X_new)[:, 1] prediction = (probability >= threshold).astype(int) results = new_data.copy() results["prediction"] = prediction results["positive_probability"] = probability results.to_csv("predictions.csv", index=False) print(f"Scored {len(results)} rows") |
Batch-prediction rule Do not silently drop required features or invent defaults simply to make a batch run succeed. A failed schema check is valuable information: it indicates that the incoming data contract no longer matches the model contract. |
A reusable batch program
A production-minded script separates the main steps into small functions. This makes schema checks, prediction behavior, and error handling easier to test independently.
Python example — reusable batch-prediction function
| def run_batch(input_csv, output_csv, pipeline, metadata): data = pd.read_csv(input_csv) X = validate_columns(data, metadata["feature_list"]) probabilities = pipeline.predict_proba(X)[:, 1] threshold = metadata.get("decision_threshold", 0.50) output = data.copy() output["prediction"] = (probabilities >= threshold).astype(int) output["positive_probability"] = probabilities output.to_csv(output_csv, index=False) return output try: scored = run_batch("new_data.csv", "predictions.csv", pipeline, metadata) except (FileNotFoundError, ValueError) as exc: print(f"Batch prediction failed: {exc}") |
39.2 Simple prediction API
A prediction API allows another program to send input data over a network and receive a model result. REST-style APIs commonly expose an endpoint such as POST /predict. The client sends a request body, the server validates it, transforms it into the model schema, generates a prediction, and returns a structured response.
| API concept | Meaning in a prediction service |
|---|---|
| REST endpoint | A named network location such as /predict that performs a defined operation. |
| Request data | Feature values supplied by a client, commonly encoded as JSON. |
| Input validation | Checking types, ranges, required fields, and allowed structure before prediction. |
| Prediction response | Structured output containing the class/value, probability or confidence, model version, and other approved fields. |
| Error handling | Returning clear client errors for invalid requests and controlled server errors for internal failures. |
| Logging | Recording operational events without exposing confidential feature values or secrets. |
API boundary Treat the API as a data contract, not just a Python function exposed to the network. The model should receive only validated inputs in the exact form expected by the persisted pipeline. |
Python example — minimal FastAPI prediction endpoint
| from fastapi import FastAPI, HTTPException from pydantic import BaseModel import pandas as pd app = FastAPI(title="Prediction API") class PredictionRequest(BaseModel): features: dict[str, float] @app.post("/predict") def predict(request: PredictionRequest): try: row = pd.DataFrame([request.features]) X = validate_columns(row, expected_features) probability = float(pipeline.predict_proba(X)[0, 1]) prediction = int(probability >= threshold) return { "prediction": prediction, "positive_probability": probability, "threshold": threshold, "model_version": metadata["version"], } except ValueError as exc: raise HTTPException(status_code=422, detail=str(exc)) from exc |
Request and response example
| Direction | Example |
|---|---|
| Client → API | POST /predict with JSON containing a features object. |
| API validation | Check required names, numeric types, permitted ranges, and request size. |
| API → model | Create a one-row DataFrame in the expected feature order. |
| Model → API | Return probability and class decision. |
| API → client | Return JSON with prediction, probability, threshold, and model version. |
Do not expose internals A client-facing error message should explain how to correct the request without revealing stack traces, filesystem paths, credentials, secret configuration, or internal model details. Keep detailed exceptions in protected server logs. |
Logging without leaking data
Python example — log operational metadata, not raw sensitive inputs
| import logging import time logger = logging.getLogger("prediction_api") def logged_prediction(X): started = time.perf_counter() probability = pipeline.predict_proba(X)[:, 1] elapsed_ms = (time.perf_counter() - started) * 1000 logger.info( "prediction_completed rows=%d latency_ms=%.2f model_version=%s", len(X), elapsed_ms, metadata["version"] ) return probability |
39.3 Simple web interface
A web interface can make a model accessible to a non-programmer. The appropriate technology depends on whether the goal is a quick internal demonstration, a small API service, or integration into a larger web system.
| Technology | Best fit for a basic ML interface | Typical characteristic |
|---|---|---|
| Streamlit | Rapid internal prototypes, teaching demos, interactive data apps. | Very little frontend code; Python-centric widgets. |
| Flask | Small custom web services or lightweight server-rendered applications. | Minimal framework; developer chooses validation and project structure. |
| FastAPI | Typed JSON APIs and backend prediction services. | Request models, validation, automatic API documentation. |
| Django | Prediction features inside a larger authenticated web platform. | Full web framework with routing, ORM, forms, users, admin, and security features. |
A small Streamlit interface
For teaching and prototypes, Streamlit provides a direct way to turn Python input widgets into a one-row DataFrame that can be passed to the saved pipeline. A real interface should add domain-specific ranges, descriptions, and validation rather than exposing raw model column names without context.
Python example — dynamically create a simple Streamlit input form
| import streamlit as st import pandas as pd st.title("Model Prediction Demo") st.caption(f"Model version: {metadata['version']}") values = {} for feature in expected_features: values[feature] = st.number_input(feature, value=0.0) if st.button("Predict"): row = pd.DataFrame([values], columns=expected_features) probability = float(pipeline.predict_proba(row)[0, 1]) prediction = int(probability >= threshold) st.metric("Positive-class probability", f"{probability:.1%}") st.write("Prediction:", prediction) |
Interface design matters Model features may be technically valid but unsuitable as direct user-interface labels. A usable application should explain units, acceptable ranges, required values, category meanings, and what the prediction does—and does not—mean. |
Choosing an interface technology
- Choose Streamlit when the primary goal is a fast interactive prototype or internal analytics tool.
- Choose FastAPI when the model must be called reliably by other software through a typed API.
- Choose Flask when a small custom web application or service needs a minimal framework and flexible structure.
- Choose Django when prediction is one feature inside a larger application that also needs authentication, database models, permissions, forms, administration, or complex workflows.
39.4 Production considerations
A demonstration that produces a prediction is not automatically production-ready. Operational deployment introduces constraints that were not visible during notebook development. These concerns should influence both the application architecture and the model selected in Chapter 32.
| Concern | Question to ask | Possible control |
|---|---|---|
| Latency | How quickly must one prediction be returned? | Measure preprocessing + inference time; use timeouts and appropriate hardware. |
| Concurrent requests | How many predictions may arrive at once? | Load testing, process/worker configuration, queueing, horizontal scaling. |
| Security | Can untrusted users reach the service or artifact? | Network controls, secure deployment, patched dependencies, trusted artifacts only. |
| Authentication | Who is allowed to request predictions? | Identity provider, API keys/tokens, role-based access where appropriate. |
| Input limits | Can extremely large or malformed requests consume resources? | Request-size limits, schema constraints, rate limits, value-range validation. |
| Logging | What must be recorded for operations and audit? | Request IDs, latency, version, status; avoid unnecessary sensitive values. |
| Privacy | Does prediction data contain personal/confidential information? | Data minimization, retention policy, access controls, encryption where required. |
| Dependencies | Can the saved model be reproduced in the serving environment? | Pinned environment, container image, tested upgrades, model metadata. |
Latency and prediction-time measurement
Python example — basic local inference timing
| from time import perf_counter started = perf_counter() probabilities = pipeline.predict_proba(X_new) elapsed = perf_counter() - started print(f"Rows: {len(X_new)}") print(f"Total inference time: {elapsed:.4f} s") print(f"Average per row: {1000 * elapsed / len(X_new):.3f} ms") |
Security reminder Never load a joblib or pickle artifact supplied by an untrusted user. Model-serving code should load only approved, controlled artifacts whose origin and integrity are known. |
Input limits and range checks
Python example — explicit operational range validation
| def validate_numeric_ranges(frame, limits): for feature, (minimum, maximum) in limits.items(): bad = ~frame[feature].between(minimum, maximum) if bad.any(): raise ValueError( f"{feature} contains values outside [{minimum}, {maximum}]" ) return frame |
A basic production-readiness checklist
- The persisted pipeline and metadata are versioned and loaded from an approved location.
- The application validates feature names, types, categories, units, ranges, and request size.
- The decision threshold and model version are explicit rather than hidden in application code.
- Prediction failures are handled predictably; detailed stack traces are not returned to users.
- Logs contain operational information needed for monitoring while respecting privacy requirements.
- Authentication, authorization, transport security, and rate limits are appropriate to the deployment context.
- Dependencies are pinned or otherwise controlled and upgrades trigger compatibility tests.
- A monitoring and rollback plan exists before the model affects important operational decisions.
Practical lab — Build a basic prediction application
Students choose one of two implementation tracks. Both tracks start from the complete model artifact created in Chapter 38 and must demonstrate schema validation, prediction, probability output, explicit error handling, and a reproducible verification step.
Required starting artifacts Use pipeline.joblib and metadata.json from Chapter 38. If those files are not available, first train and persist a complete pipeline using the Chapter 38 lab. |
Common setup for both tracks
Lab setup — load the model contract once
| from pathlib import Path import json import joblib import pandas as pd artifact_dir = Path("artifacts/model_v1") pipeline = joblib.load(artifact_dir / "pipeline.joblib") with (artifact_dir / "metadata.json").open("r", encoding="utf-8") as file: metadata = json.load(file) features = metadata["feature_list"] threshold = metadata.get("decision_threshold", 0.50) |
Track A — Batch prediction program
1. Create a new_data.csv file containing at least ten observations and the expected feature columns.
2. Read the CSV and validate missing and unexpected columns.
3. Generate probabilities and threshold-based predictions using the saved pipeline.
4. Add prediction, probability, model version, and threshold columns to the result table.
5. Export predictions.csv and confirm the output row count matches the input row count.
6. Deliberately remove one required column and demonstrate that the program fails with a clear error instead of scoring the malformed input.
Track A — complete batch-scoring function
| def score_csv(input_path, output_path): data = pd.read_csv(input_path) X = validate_columns(data, features) probability = pipeline.predict_proba(X)[:, 1] result = data.copy() result["prediction"] = (probability >= threshold).astype(int) result["positive_probability"] = probability result["model_version"] = metadata["version"] result["decision_threshold"] = threshold result.to_csv(output_path, index=False) return result result = score_csv("new_data.csv", "predictions.csv") assert len(result) == len(pd.read_csv("new_data.csv")) |
Track B — Interactive Streamlit interface
1. Create a Streamlit script named app.py.
2. Display the model version and prediction threshold.
3. Create an input widget for every expected feature. Add units/range guidance where known.
4. Convert the entered values into a one-row DataFrame in the expected feature order.
5. When the user selects Predict, display the predicted class and positive-class probability.
6. Handle invalid input with a clear message and do not expose Python stack traces in the interface.
Track B — basic interactive prediction interface
| import streamlit as st import pandas as pd st.set_page_config(page_title="Prediction Demo") st.title("Interactive Prediction Demo") st.write(f"Model: {metadata['model_name']} — {metadata['version']}") values = { feature: st.number_input(feature, value=0.0) for feature in features } if st.button("Predict", type="primary"): try: row = pd.DataFrame([values], columns=features) probability = float(pipeline.predict_proba(row)[0, 1]) prediction = int(probability >= threshold) st.success(f"Prediction: {prediction}") st.metric("Positive-class probability", f"{probability:.1%}") except (ValueError, TypeError) as exc: st.error(f"Input could not be scored: {exc}") |
Verification requirements
| Check | Evidence students should provide |
|---|---|
| Artifact consistency | Show the loaded model version and threshold from metadata. |
| Valid prediction | One valid input produces a prediction and probability. |
| Schema/input protection | One intentionally malformed input is rejected clearly. |
| Reproducibility | Running the same valid input twice produces the same result for a deterministic saved artifact. |
| Output contract | Document the fields returned to the CSV or interface. |
| Operational reflection | Identify at least three controls still needed before real production use. |
Student deliverable
- Source code for the selected track.
- A short README explaining how to install dependencies and run the program.
- A screenshot or sample output showing a successful prediction.
- Evidence of one rejected invalid input.
- A short paragraph explaining how preprocessing consistency is preserved.
- A short production-readiness note covering security, privacy, logging, and dependency management.
Knowledge check
1. Why should a prediction application load the complete pipeline instead of reproducing preprocessing manually?
2. What is the difference between an invalid request and a valid request that receives a low-confidence prediction?
3. Why should an API return a controlled error rather than a full Python traceback?
4. What metadata should a prediction response or log include to make model versions traceable?
5. When would batch prediction be preferable to an API?
6. Why is a Streamlit demonstration not automatically production-ready?
7. Name four production controls that are unrelated to the model's predictive accuracy.
Chapter summary
| Concept | Key takeaway |
|---|---|
| Batch prediction | Validate the input contract, reuse the persisted pipeline, score the data, and export traceable results. |
| Prediction API | A request-response boundary requires validation, controlled errors, logging, and a stable output contract. |
| Web interface | Choose the framework according to the application context; usability and input guidance are part of model safety. |
| Production concerns | Latency, concurrency, security, authentication, privacy, limits, logging, and dependency control affect deployment quality. |
| Practical principle | A useful prediction application is not only able to predict; it also rejects invalid inputs, exposes the model version, and behaves predictably when something goes wrong. |
Next step After a prediction application exists, the machine-learning lifecycle shifts from offline evaluation to ongoing operation. The next topics should address monitoring, drift, retraining triggers, model/version governance, and safe maintenance after deployment. |