1. ML Model Testing Fundamentals(High Priority)
✅ Understand these concepts deeply:
What is F1-score, precision, recall, AUC, accuracy – and how they differ.
How to test ML models: unit testing, integration testing, black-box testing, bias/fairness testing, adversarial testing.
Model versioning, reproducibility & traceability.
📚 Resources:
https://madewithml.com/ → “Evaluation” and “Testing” sections
https://github.com/eugeneyan/testing-ml → ML testing techniques repo
🧠 Be able to answer:
How do you test a classification/regression model end-to-end?
How do you validate a model in production?
How do you detect drift or bias?
2. MLOps & Tooling Knowledge
✅ Cover basics of:
MLflow: Tracking experiments, model registry, deployment stages.
Great Expectations: Data quality assertions.
CI/CD for ML: How to integrate tests (unit tests + model checks) in GitHub Actions/Jenkins pipelines.
📚 Learn from:
https://mlflow.org/docs/latest/index.html
https://docs.greatexpectations.io/docs/tutorials/intro/
Google: “CI/CD for ML GitHub Actions sample” or “MLOps pipeline testing”
🧠 Be able to answer:
How do you test a model before deployment?
What tests would you include in a CI/CD pipeline for ML models?
How to validate input data in an ML pipeline?
3. Python + Pytest for ML Testing
✅ Brush up quickly on:
Writing unit tests for Python functions (data preprocessing, model code).
Pytest + mocking basics (mocking external APIs, random inputs).
Test structure: setup, assert, fixtures, parameterize.
📚 Use:
https://realpython.com/pytest-python-testing/
https://docs.pytest.org/en/stable/contents.html
🧠 Practice questions:
Write pytest test cases for a sklearn model prediction function.
How would you mock a database or S3 call?
4. Bias, Fairness & Responsible AI (Brief but important)
✅ Understand the basics of:
What is SHAP, LIME? → Model explainability
What is bias in datasets vs. model bias?
How fairness is tested: group fairness, equal opportunity, disparate impact.
📚 Review:
https://github.com/Trusted-AI/AIF360
https://christophm.github.io/interpretable-ml-book/shap.html
🧠 Interview Qs:
How do you check if your model is biased?
How do SHAP or LIME help in explainability?
5. System Testing + Performance Metrics
✅ Learn how to:
Test ML APIs (Rest API with model behind).
Monitor latency, throughput, drift detection, A/B testing setups.
📚 Tools:
Prometheus + Grafana basics
Locust for load testing ML APIs
Google: “Test strategy for AI/ML systems”
📌 Cheat Sheet Links
