Memory hook: Experiment freely; promote with evidence.
Must remember
- Track runs with MLflow, including parameters, metrics, artifacts and source/data versions. Notebooks support exploration; training scripts and components support repeatable jobs.
- AutoML explores supported model choices; hyperparameter sweeps optimize configured training choices. Distributed training requires compatible frameworks, data partitioning and sufficient communication bandwidth.
- Build pipelines for preparation, training and evaluation. Avoid leakage, use appropriate validation splits and compare models against an agreed business baseline.
- Register the model with its runtime and feature retrieval/preprocessing specification. A model artifact without compatible features and dependencies is not a deployable solution.
- Managed online endpoints suit low-latency requests; batch endpoints suit asynchronous collections. Test authentication, network paths, input schema, resource limits and failure behavior.
- Progressive rollout, traffic allocation and known-good versions support safe rollback. Responsible AI evaluation and release gates should precede promotion; archive superseded artifacts according to policy rather than losing lineage.
Choose under exam pressure
| Requirement | Choice and reason |
|---|---|
| Nightly scoring of a large dataset | A batch endpoint/workflow with completion and correctness checks. |
| Interactive prediction with a strict latency target | A sized online endpoint and measured tail latency. |
Traps
- The highest validation score is not sufficient if the model violates latency, fairness or cost requirements.
- Registering a model does not automatically deploy it.
Active recall
1. What does MLflow tracking preserve?
Experiment parameters, metrics and artifacts linked to a run.
2. Why package feature definitions?
Serving must compute or retrieve features consistently with training.
3. What distinguishes hyperparameters from weights?
Hyperparameters configure training; weights are learned during training.
4. How safely replace a model?
Evaluate, deploy a limited traffic share, monitor defined gates and retain rollback.
5. Why keep archived model lineage?
To explain past predictions and reproduce or investigate earlier releases.