Advanced 75 Questions • ~15 hrs study

Deep Learning (Q101-175)

Neural networks, CNNs, RNNs, Transformers, backpropagation, optimization algorithms, and training techniques for deep models.

101

Explain the vanishing gradient problem and how to solve it.

Short Answer

In deep networks, gradients become exponentially small as they backpropagate through layers (especially with sigmoid/tanh), making early layers learn very slowly or not at all.

Solutions

  • ReLU activation (gradients don't shrink for positive values)
  • Residual connections (skip connections in ResNet)
  • Batch Normalization (keeps activations in good range)
  • Proper weight initialization (He, Xavier/Glorot)
  • LSTM/GRU for recurrent networks (gating mechanism)

💡 Memory Trick: "Vanishing = signal fading over distance. Fix: shortcuts (ResNet), better activations (ReLU), normalization"

102

What is the Transformer architecture and why did it replace RNNs?

Short Answer

Transformers use self-attention to process all positions in parallel (no sequential dependency like RNNs). This enables faster training, better long-range dependency capture, and massive scaling.

Key Components

  • Multi-Head Self-Attention: Each head learns different relationships
  • Positional Encoding: Injects sequence order information
  • Feed-Forward Network: Applied position-wise
  • Layer Normalization + Residual Connections

💡 Memory Trick: "Transformer = Attention Is All You Need. RNN reads word-by-word, Transformer reads the whole page at once"

103

Compare Adam, SGD, and AdamW optimizers.

Short Answer

SGD: Simple, good generalization but slow convergence. Adam: Adaptive learning rates (momentum + RMSprop), fast convergence. AdamW: Adam with decoupled weight decay, better generalization than Adam.

Optimizer Pros Use When
SGD + MomentumBest generalizationCNNs, well-tuned setups
AdamFast convergence, less tuningNLP, quick experiments
AdamWAdam + proper weight decayTransformers, LLMs

💡 Memory Trick: "SGD = Simple & steady | Adam = Adaptive & fast | AdamW = Adam done right"

Questions 104-175 continue with the same detailed format.

Topics covered: CNN architectures (ResNet, VGG, EfficientNet), RNN/LSTM/GRU, Batch/Layer/Group Normalization, Dropout, Learning Rate Scheduling, Transfer Learning, Knowledge Distillation, Mixed Precision Training, Distributed Training, and more.

View Full Documentation →
Previous: Fundamentals (Q1-100) Next: NLP, LLMs & RAG (Q176-250)