Implementing Machine Learning Models in Your App

The Notebook-to-Production Gap

Most ML projects fail not at modelling but at productionisation. Getting from a demo to a system that serves predictions reliably, observably, and within budget is its own engineering discipline.

The MLOps Lifecycle

1. Feature Engineering

Use a feature store (Feast, Tecton) when features are reused across models or need point-in-time correctness. For single-model use cases, well-modelled tables in your warehouse are enough.

2. Training

• Track every experiment (MLflow, Weights & Biases)

• Version data, code, and model artefacts together

• Make training reproducible from a single command

• Use the same code path for training and retraining

3. Validation

• Hold-out and time-based splits

• Bias and fairness checks on protected attributes

• Performance on slices, not just averages

• Compare against the current production model on the same data

4. Deployment

• Containerised model servers (TorchServe, BentoML, or custom FastAPI)

• Canary rollouts with traffic shifting

• Shadow traffic for risk-free comparison

• Auto-rollback on error rate or latency regressions

5. Monitoring

• Data drift (feature distribution changes)

• Concept drift (label distribution changes)

• Performance metrics tied to business outcomes

• Latency and cost per inference

6. Retraining

Automated retraining when drift exceeds thresholds — but always with a human-in-the-loop validation step before promoting a new model.

Architectural Choices

• Online vs. batch. If predictions can be precomputed daily, do that. It's an order of magnitude cheaper.

• Real-time vs. near-real-time. Real-time pricing or fraud needs sub-100ms inference. Recommendations can often run async.

• CPU vs. GPU. Most classical ML and small NLP models are fine on CPU. Reach for GPUs when latency or model size demands it.

The Bottom Line

The model is 20% of the work. The other 80% is the platform that lets you ship it, observe it, and replace it without drama.

*We build ML platforms that turn experiments into reliable production systems. Talk to us →*