USA
Canada
UK
Europe
Australia
Singapore
Japan
IndiaQ1. How does a transformer work at a high level?
Short answer: Self-attention lets each token weigh all others, capturing context in parallel.
Strong answer: Transformers use self-attention so every token attends to every other token, weighting relevance, combined with feed-forward layers and positional information. This captures long-range context and trains in parallel (unlike RNNs). Modern LLMs are large decoder-style transformers trained to predict the next token.
Likely follow-up: Why did transformers replace RNNs for language?
What the interviewer is checking: Architecture fundamentals.
Common mistake: Describing LLMs as simple lookup or Markov models.
Q2. What is the difference between fine-tuning, RAG and prompting?
Short answer: Prompting steers, RAG adds external knowledge, fine-tuning changes behavior/format.
Strong answer: Prompting shapes output with instructions/examples at inference. RAG injects retrieved, up-to-date or proprietary context so the model grounds answers in your data. Fine-tuning updates weights to change tone, format or domain behavior. Start with prompting, add RAG for knowledge, fine-tune for behavior when prompt+retrieval isn’t enough.
Likely follow-up: Why not fine-tune to add frequently-changing facts?
What the interviewer is checking: Practical LLM strategy.
Common mistake: Fine-tuning to inject knowledge that changes daily.
Q3. What is a context window and why does it matter?
Short answer: The max tokens a model can consider at once; it bounds input + output.
Strong answer: The context window is the token budget for prompt plus generation. It limits how much document/history you can include, drives cost and latency, and forces strategies like chunking, retrieval and summarization for large inputs. Bigger windows help but don’t eliminate the need for good retrieval and context management.
Likely follow-up: How do you handle a document larger than the context window?
What the interviewer is checking: Practical LLM constraints.
Common mistake: Assuming you can just paste unlimited text.
Q4. How do you control LLM output cost and latency?
Short answer: Right-size the model, cap tokens, cache, batch, and use retrieval to shorten prompts.
Strong answer: Choose the smallest model that meets quality, cap max tokens, cache repeated prompts/responses, stream output, batch where possible, and keep prompts lean via good retrieval instead of stuffing context. Monitor tokens per request. For scale, route easy queries to cheaper models.
Likely follow-up: How does prompt size affect cost?
What the interviewer is checking: Production LLM economics.
Common mistake: Sending huge prompts to the largest model for every request.
Q5. How do you evaluate an LLM or LLM feature?
Short answer: Task-specific metrics, groundedness checks, and human or LLM-as-judge review.
Strong answer: Define success per task (correctness, groundedness, relevance, safety, format), build eval sets, and combine automated metrics, LLM-as-judge (carefully) and human review for high-stakes cases. Track regressions across prompt/model changes like tests. Vibe-checking alone is not evaluation.
Likely follow-up: How do you catch hallucinations systematically?
What the interviewer is checking: Evaluation rigor.
Common mistake: Shipping on anecdotal "looks good" testing.
Q6. What causes hallucinations and how do you reduce them?
Short answer: Models predict plausible text; ground with retrieval, constrain, cite and verify.
Strong answer: Hallucinations arise because LLMs generate statistically plausible text, not verified facts. Reduce them by grounding in retrieved authoritative context, instructing the model to say "I don’t know", requiring citations, constraining to schemas, lowering temperature for factual tasks, and adding a verification pass for high-stakes output.
Likely follow-up: How would you make a model abstain when it lacks context?
What the interviewer is checking: Reliability engineering.
Common mistake: Assuming a bigger model removes hallucination.
Q7. What are temperature and top-p, and how do they affect output?
Short answer: They control randomness/diversity of generation.
Strong answer: Temperature scales the probability distribution — lower is more deterministic/focused, higher is more creative/varied. Top-p (nucleus) samples from the smallest set of tokens whose cumulative probability exceeds p. Use low temperature for factual/structured tasks and higher for creative ones. Tune per use case.
Likely follow-up: What temperature would you use for a data-extraction task?
What the interviewer is checking: Practical generation control.
Common mistake: Using high temperature for tasks needing consistency.