knaxuslearn · build · ship

Roadmaps / AI / Interview topics

Reference

Interview topics

Use these for AI engineering and backend roles. Each follow-up is the question a good interviewer asks next. The Staff line is what lifts the answer.

Phase 1

QuestionFollow-upA Staff answer adds
How do you cut the cost of an AI feature?What do you give up?A measured table of cost against quality, caching rules and a router
Where does latency come from?How do you reduce it?Time to first token against generation, streaming and shorter inputs
How do you handle a model timeout?What about retries?Backoff with jitter, idempotency and a cap on total spend
Hosted or self-served model?How do you decide?Volume, latency, data rules and a break-even calculation

System design prompt (20 minutes): a summarisation feature for 1 million documents a day. Outline: numbers and budget (3 min), model choice and batching (5), failure handling (4), quality checks (4), observability (4).

Phase 2

QuestionFollow-upA Staff answer adds
How do you know the system got better?What if the score changes by 2 points?Spread over runs, a noise floor and a paired comparison
How do you build an evaluation set?How do you keep it honest?Coverage, unanswerable cases, a second labeller and a held-out part
Can a model judge another model?When does it fail?Agreement with human labels, bias and a re-check after every change
How do you stop regressions?What runs in CI?A fast subset in CI, a full run nightly and a rule for failing a build

System design prompt (20 minutes): an evaluation pipeline for a customer-facing assistant. Outline: what to measure (3 min), data and labels (5), scorers (5), CI and monitoring (4), review process (3).

Phase 3

QuestionFollow-upA Staff answer adds
How does retrieval-augmented generation fail?How do you find the cause?A failure list by cause, recall at k and a fix with a measurement
Vector, keyword or hybrid search?When do you add a reranker?A comparison on your questions and the latency cost
How do you cite sources?How do you check a citation?Passage ids, a checker and a refusal rule
How do you handle access control?Where do you filter?In the retrieval query, with a test for a second user

System design prompt (20 minutes): a company knowledge assistant with per-team access. Outline: requirements and numbers (3 min), ingestion and chunking (5), retrieval and permissions (5), answer generation and citations (4), freshness and cost (3).

Phase 4

QuestionFollow-upA Staff answer adds
When do you use an agent?When do you not?A workflow by default, step limits and a measured reason for the loop
How do you defend against prompt injection?Can a filter solve it?Least privilege, confirmations, an allow-list and a suite of attacks
How do you design a tool server?What about authorisation?Narrow tools, validated arguments, per-request identity and audit logs
How do you upgrade a model?What can break?An evaluation comparison, an injection re-run and a staged rollout

System design prompt (20 minutes): an agent that can refund small orders. Outline: limits and risk (3 min), tools and confirmations (5), injection defences (5), audit and monitoring (4), rollout (3).