Advanced 20 Questions • ~10 hrs study

System Design for ML Engineers

Learn to design scalable ML systems — recommendation engines, search ranking, fraud detection, real-time inference, and data pipelines.

1

Design a real-time fraud detection system.

Short Answer

Multi-layer approach: Rule-based (fast, catches known patterns) → ML model (gradient boosting for tabular features) → Real-time streaming (Kafka + Flink) → Human review queue for borderline cases.

Key Components

  • Feature Store: Real-time (last 5 min aggregates) + Batch (historical patterns)
  • Model: Ensemble of rules + XGBoost + deep learning for sequences
  • Latency: Must decide in < 100ms for payment authorization
  • Feedback loop: Label from chargebacks (delayed by 30-90 days)
  • Monitoring: False positive rate, approval rate, model drift

💡 Memory Trick: "Fraud detection = Rules (fast) + ML (smart) + Streaming (real-time) + Human (final check)"

2

Design a search ranking system (like Google/Bing).

Short Answer

Multi-stage ranking: Query understanding → Retrieval (BM25 + semantic search) → Ranking (learning-to-rank with BERT-based cross-encoder) → Re-ranking (personalization, freshness, diversity).

Architecture

  • Query Processing: Spelling correction, query expansion, intent detection
  • Retrieval: Inverted index (BM25) + ANN (dense embeddings) → top 1000
  • Ranking: Cross-encoder scores query-document pairs → top 100
  • Features: Click-through rate, page authority, freshness, user signals
  • Metrics: NDCG, MRR, user satisfaction (dwell time, pogo-sticking)

💡 Memory Trick: "Search = Funnel: Billions of docs → 1000 (retrieval) → 100 (ranking) → 10 (shown)"

3

How would you design an ML pipeline for continuous training?

Short Answer

Continuous training pipeline: Data ingestion (scheduled + event-driven) → Validation (schema + distribution checks) → Training (automated with hyperparameter tuning) → Evaluation (against champion model) → Deployment (canary/blue-green).

Components

  • Orchestration: Airflow, Kubeflow Pipelines, Vertex AI
  • Feature Store: Feast, Tecton (point-in-time correctness)
  • Experiment Tracking: MLflow, Weights & Biases
  • Model Registry: Version control for models
  • Trigger: Time-based (daily) or drift-based (auto-retrain)

💡 Memory Trick: "ML Pipeline = Data → Validate → Train → Evaluate → Deploy → Monitor → Repeat"

Questions 4-20 continue with system design problems.

Topics covered: Notification System, Content Moderation at Scale, Real-time Recommendation, Ad Click Prediction, Video Processing Pipeline, Chat System with AI, and designing for low-latency inference.

View Full Documentation →
Back to Topics Next: Mock Interviews