Advanced 75 Questions • ~15 hrs study

NLP, LLMs & RAG (Q176-250)

Natural Language Processing, Large Language Models, and Retrieval-Augmented Generation — tokenization, attention, fine-tuning, prompt engineering, and RAG architectures.

176

What is the difference between BERT and GPT?

Short Answer

BERT is a bidirectional encoder (sees full context) — great for understanding tasks. GPT is an autoregressive decoder (left-to-right) — great for generation. BERT = understanding, GPT = generation.

Aspect BERT GPT
ArchitectureEncoder onlyDecoder only
DirectionBidirectionalLeft-to-right
TrainingMasked LM + NSPNext token prediction
Best ForClassification, NER, QAGeneration, chat, code

💡 Memory Trick: "BERT = Reads the whole sentence at once (fill-in-the-blank) | GPT = Writes one word at a time (autocomplete)"

177

What is RAG and why is it important for LLMs?

Short Answer

Retrieval-Augmented Generation combines a retriever (fetches relevant documents) with a generator (LLM). It grounds LLM responses in factual data, reduces hallucinations, and enables knowledge updates without retraining.

RAG Pipeline

  • Indexing: Chunk documents → embed → store in vector DB
  • Retrieval: Embed query → find similar chunks → top-k results
  • Generation: Inject retrieved context into LLM prompt
  • Evaluation: Faithfulness, relevance, answer correctness

💡 Memory Trick: "RAG = Open-book exam for LLMs. Instead of memorizing everything, look it up in your notes"

178

Explain LoRA and QLoRA for efficient fine-tuning.

Short Answer

LoRA (Low-Rank Adaptation) freezes pretrained weights and injects trainable low-rank matrices into attention layers. QLoRA adds 4-bit quantization to further reduce memory. Fine-tune 7B+ models on a single GPU.

Key Points

  • LoRA: W = W₀ + BA (B and A are low-rank, trainable)
  • Rank r=8-64 typically sufficient
  • QLoRA: 4-bit NormalFloat quantization + LoRA
  • Memory: 7B model from 28GB → ~6GB with QLoRA

💡 Memory Trick: "LoRA = Adding sticky notes to a textbook instead of rewriting it. QLoRA = compressed textbook + sticky notes"

Questions 179-250 continue with the same detailed format.

Topics covered: Tokenization (BPE, WordPiece, SentencePiece), Word Embeddings (Word2Vec, GloVe, FastText), Attention mechanisms, Prompt Engineering, Chain-of-Thought, Function Calling, Agents, Vector Databases, Chunking Strategies, Evaluation (BLEU, ROUGE, BERTScore), and more.

View Full Documentation →
Previous: Deep Learning (Q101-175) Next: Computer Vision (Q251-290)