THE FDE CURRICULUM
Understand the concepts before you drill the questions
A structured path through the ideas Forward Deployed and Applied AI loops actually test. Each concept gives you the intuition, a worked example, and the trade-off interviewers probe, then links straight to the real questions where it shows up. Read it like a curriculum, or jump to whatever you are weakest on.
114 concepts across 10 tracks · foundational concepts are free
Begin the curriculum →01
🧠 Foundations of LLMs & GenAI
How language models actually work: tokens, attention, context, sampling, and the prompting-vs-RAG-vs-fine-tuning decision every loop opens with.
START HERE
01Tokenization & TokensFree
A language model does not read characters or words. It reads tokens: sub-word chunks produced by a tokenizer, each mapped to an integer the model embeds. Tokens are the unit of the context window and of billing, and the way text splits into them explains a surprising number of model quirks, which is why almost every loop opens here.02The Context WindowFree
The context window is the fixed number of tokens a language model can attend to at once, and input and output share that same budget. Understanding it is what separates engineers who can size a prompt, control cost and latency, and decide when to reach for RAG from those who just paste everything in and hope.03Embeddings & Vector RepresentationsFree
An embedding turns a piece of text into a list of numbers positioned so that similar meanings land near each other in space, which lets you search by meaning instead of by keyword. Embeddings are the engine under RAG, semantic search, clustering, and deduplication, so FDE loops expect you to explain cosine similarity and the pitfalls that quietly break a vector index.04The Transformer, IntuitivelyFree
The transformer is the architecture behind every modern large language model, built on self-attention that lets each token look at every other token in parallel. FDE loops do not want the math; they want you to explain why attention beat RNNs, what decoder-only means, and why context length is expensive, in plain language an exec or a teammate can follow.05Attention and Self-AttentionCore
Attention computes a weighted sum of value vectors, where the weights come from how well each token's query matches every other token's key. Self-attention applies this within one sequence so each token can pull from all the others, and the all-pairs comparison is why cost grows with the square of sequence length. FDE loops probe it because it explains context limits, latency, and the KV cache in one mental model.06RoPE and Positional EncodingsCore
Self-attention has no built-in sense of word order, so transformers inject position information into the token vectors. Rotary position embeddings (RoPE) rotate query and key vectors by a position-dependent angle so relative position falls out of the dot product, which is why RoPE underpins almost every long-context model and why extending a context window means rescaling RoPE.07Temperature, Top-p and SamplingFree
At each step a model outputs a probability over every possible next token, and sampling settings like temperature, top-p, and top-k decide how that distribution is turned into an actual choice. FDE loops test this because it controls the determinism-versus-creativity dial: knowing when to set it low for extraction and high for brainstorming, and why the same prompt giving different answers is expected, not a bug.08Constrained DecodingCore
Constrained decoding forces a model's output to match a schema or grammar by masking the logits at each step so the model can only sample tokens the grammar still allows. It guarantees structurally valid output (JSON, SQL, a fixed enum) at a small latency cost, which is why FDE loops reach for it the moment a pipeline depends on parseable model output.09Prompt EngineeringFree
Prompt engineering is the practice of shaping a model's behavior through the instructions, examples, and format constraints you give it, before reaching for retrieval or fine-tuning. FDE loops test it because it is the cheapest, fastest lever you have, and a candidate who can make a model reliable with a well-structured prompt has saved a project weeks of unnecessary infrastructure.10Chain-of-Thought PromptingCore
Chain-of-thought prompting tells a model to write out intermediate reasoning steps before its final answer, which raises accuracy on multi-step math, logic, and planning by spending more compute per problem. FDE loops probe it because knowing when it helps, when it just burns tokens, and why the stated reasoning is not always the true cause of the answer separates people who have shipped from people who have read a blog post.11Why LLMs HallucinateFree
An LLM generates the most plausible next token given its training, with no built-in notion of truth or any source to check against, which is why it can produce confident, fluent, and completely fabricated answers. FDE loops test this because every enterprise buyer asks 'can we trust it,' and you need to explain the cause and the mitigations (grounding, refusal, citations, evals) in terms an exec will accept.12Fine-tuning vs RAG vs PromptingFree
Prompting, RAG, and fine-tuning are the three ways to adapt a model to your problem, and choosing among them is the decision FDE interviewers probe most. The framework: prompt first, add RAG when the model needs facts it lacks or must cite, and fine-tune to change behavior or format rather than knowledge. They compose; they are not rivals.13RLHF (Alignment)Core
RLHF aligns a model to human preferences in three stages: supervised fine-tuning on demonstrations, training a reward model from human comparisons of outputs, then optimizing the policy with RL against that reward while a KL penalty anchors it to the base model. It shapes behavior and tone rather than facts, and FDE loops probe it because reward hacking and the KL anchor are where deployments actually go wrong.14Reward ModelsCore
A reward model scores a candidate output by how much a human would prefer it, learned from pairwise comparisons rather than absolute ratings. It is the signal that drives RLHF, ranks best-of-N samples, and guides test-time search, which is why FDE loops probe how it is trained, where it leaks, and the outcome-versus-process distinction.15Constitutional AI and RLAIFCore
Constitutional AI aligns a model against a written set of principles using AI-generated feedback instead of mostly human labels: the model critiques and revises its own outputs against the principles, then learns from an AI judge that picks which response follows them better. RLAIF scales where human labeling stalls, which is why FDE loops probe what the constitution actually encodes and whose biases the AI judge inherits.16KV CacheCore
During autoregressive decoding a model would recompute attention over every prior token at each step; the KV cache stores each token's key and value vectors so each new token only attends, never recomputes, making per-token generation far cheaper. The cache grows with sequence length times batch size and becomes the memory bottleneck in serving, which is what motivates PagedAttention. FDE loops probe it because it explains why long contexts are costly to serve and why throughput, not the model, is often the constraint.17LoRA and Parameter-Efficient Fine-tuningPremium
Full fine-tuning updates every weight in a model, which is expensive to train and produces a full-size checkpoint per task. LoRA freezes the base model and trains small low-rank adapter matrices instead, giving tiny swappable checkpoints; QLoRA adds a quantized frozen base so the whole thing fits on a single GPU. FDE loops probe it because it is how you adapt a model on a customer's data without their budget or their hardware blowing up.18Direct Preference Optimization (DPO)Premium
DPO aligns a model directly from preference pairs (chosen vs rejected) without training a separate reward model or running an RL loop. It derives a closed-form solution to the same KL-constrained objective RLHF optimizes, turning alignment into a simple classification-style loss on the log-ratio between your policy and a frozen reference. FDE loops probe it because it is the practical default for preference tuning, and the trade-offs against PPO-based RLHF are where the judgment lives.19Policy Optimization: PPO and GRPOCore
PPO and GRPO are the reinforcement-learning optimizers that turn a reward signal into model weight updates during RLHF and reasoning training. PPO uses a clipped objective and a KL anchor to keep updates stable and close to the reference model; GRPO drops the learned value network and instead normalizes rewards across a group of sampled answers. FDE loops probe these because the KL anchor, reward hacking, and GRPO's cost savings are where reasoning pipelines actually succeed or break.20Mixture of Experts (MoE)Premium
An MoE replaces the dense feed-forward block of a transformer with many parallel expert blocks plus a small router that activates only a few experts per token. This decouples total parameter count from per-token compute: the model can hold hundreds of billions of parameters while doing the work of a much smaller one on any given token. FDE loops probe it because the headline 'huge but cheap' hides a brutal serving cost, every expert must sit in memory even though most stay idle.21Scaling LawsCore
Scaling laws say a model's loss falls as a smooth power law in its parameter count, training data, and compute, so you can predict the payoff of a bigger run before you make it. The Chinchilla result showed that for a fixed compute budget you should grow data and parameters together (roughly 20 tokens per parameter), revealing that many early giant models were badly undertrained. FDE loops test this because it governs every model-size and token-budget decision.22Inference-Time ComputePremium
Inference-time compute trades extra computation at answer time, not training time, for higher accuracy: sample many solutions and pick the best, or search over reasoning steps guided by a reward model. It reshapes serving economics because cost now scales with how hard a query is, which is why FDE loops probe the accuracy-per-token trade-off and how you cap a token budget.23Multimodal Models and VLMsCore
A vision-language model lets an LLM see by running images through a vision encoder and a projection layer that turns them into tokens the language model reads alongside text. CLIP-style contrastive training aligns image and text into one embedding space, powering image search and zero-shot classification. FDE loops probe this because document, chart, and screenshot understanding is a common deployment, and the failure modes (counting, fine detail, hallucinated visual facts) are specific.24Diffusion ModelsCore
Diffusion models generate images, audio, and video by learning to reverse a gradual noising process: they start from pure noise and denoise step by step into a sample, steered by a text prompt through cross-attention. Latent diffusion runs this in a compressed space for speed, and classifier-free guidance trades diversity for prompt adherence. FDE loops probe diffusion because the steps-versus-quality-versus-latency trade-off and how it differs from autoregressive generation come up in any media-generation deployment.25Speech and Voice AICore
A voice agent chains automatic speech recognition (ASR) to transcribe audio, an LLM to decide the reply, and text-to-speech (TTS) to speak it, with voice activity detection and barge-in handling the turn-taking. Hitting a sub-second feel depends on streaming every stage and starting speech before the LLM finishes. FDE loops probe this because the latency budget across stages, and when to abandon the cascade for end-to-end speech-to-speech, is where voice deployments live or die.26Autoregressive DecodingFree
LLMs generate one token at a time: each step feeds the whole sequence back in to predict the next token, which is why generation is sequential and cannot be parallelized the way a forward pass over a known prompt can. This splits inference into a parallel prefill phase and a sequential, memory-bound decode phase, and it is why the KV cache exists and why long outputs cost what they cost.02
🤖 Retrieval & Agents
Retrieval-augmented generation end to end, vector search, reranking, and tool-using agents: the modal FDE design round.
01Retrieval-Augmented Generation (RAG)Free
RAG grounds a language model in your own data by retrieving relevant passages at query time and putting them in the prompt, so the model answers from real sources instead of memory. It is the default pattern for almost every enterprise FDE deployment, which is why nearly every loop tests it.02Vector DatabasesFree
A vector database stores embeddings alongside metadata and answers nearest-neighbor queries fast using approximate indexes. The real interview question is not how they work but when you actually need one instead of a library or plain Postgres with pgvector.03Hybrid Search (Lexical + Vector)Core
Hybrid search runs a keyword retriever (BM25) and a dense vector retriever side by side, then merges their result lists, because each one misses cases the other catches. Vectors lose exact codes and rare jargon, BM25 loses paraphrase, and combining them with Reciprocal Rank Fusion usually beats either alone.04Chunking StrategiesCore
Chunking is how you split documents into the units you embed and retrieve, and it quietly sets the recall ceiling for your entire RAG system. Get the size, boundaries, and metadata wrong and no reranker or prompt can recover the answer that never got retrieved.05Reranking and Two-Stage RetrievalCore
Two-stage retrieval pairs a cheap recall-heavy first stage that pulls dozens of candidates with a precise reranker that re-scores each one against the query. It is the standard fix when vector search returns relevant-ish chunks but the right one is not in the top few, and it trades a little latency for a lot of precision.06Approximate Nearest Neighbor (ANN)Core
Brute-force vector search is O(N*d) per query and falls apart at millions of vectors, so ANN trades a sliver of recall for orders-of-magnitude speed. The two dominant families are IVF (cluster then probe nearby cells) and HNSW (walk a navigable graph), with product quantization to shrink memory. The non-negotiable habit is measuring recall@k against a brute-force baseline.07AI Agents and Tool UseFree
An agent is a language model wrapped in a loop that lets it choose tools, act, observe the result, and decide what to do next. The skill interviewers test is judgment: knowing when that loop earns its unpredictability and when a fixed pipeline is cheaper, faster, and safer.08Tool / Function CallingCore
Tool calling is how a language model reaches outside itself: it emits a structured request (a tool name plus JSON arguments) that your application executes and feeds back. The model is only as capable as the tools you give it and as reliable as your validation, because the arguments it produces are model output and must never be trusted blindly.09Multi-Agent OrchestrationCore
Multi-agent orchestration coordinates several specialized agents (planner, worker, critic, or researcher and writer) toward one goal, deciding how they communicate, how the flow is structured, and when they stop. The hard judgment is when splitting genuinely helps versus when a single well-prompted agent is simpler and far more reliable.10GuardrailsCore
Guardrails are the layered deterministic checks you wrap around a non-deterministic model: input validation, output filtering, schema enforcement, confidence thresholds with refusal, and human approval for high-stakes actions. The principle is to put controls you can fully trust around a core you cannot, so the system stays safe even when the model misbehaves.11Agent MemoryPremium
Agent memory is how an agent carries state across turns and sessions. Short-term memory is the conversation and scratchpad living inside the context window, bounded and expensive. Long-term memory is an external store the agent writes to and retrieves from on demand, usually via RAG, so it can recall facts from last week without holding them in the prompt. FDE loops probe this because the hard parts, summarization, what to persist, and stale or contradictory memory, are where agents quietly break.12TF-IDF and BM25Free
BM25 is the lexical scoring function that still beats a lot of fancier setups out of the box. It scores a document by how often the query's terms appear (term frequency), discounted by how common those terms are across the corpus (inverse document frequency), with two refinements: frequency saturation so a term repeated 50 times does not score 50x, and length normalization so long documents do not win by sheer size. It is the backbone of the lexical half of hybrid search.13Context Window Management for FDE AgentsFree
Managing the context window is the discipline of deciding what an agent sees on every step, run against a customer's private data and their token bill. Forward Deployed Engineers treat the window as a budget to allocate with four moves, write, select, compress, and isolate, because a deployment that ignores it is either too expensive to run or too unreliable to trust.03
📊 Evaluation & ML Foundations
The metrics and methods that tell you a system works: precision/recall, eval sets, LLM-as-judge, and the classical ML still tested.
01Information Theory for ML: Entropy, Cross-Entropy, KL and PerplexityCore
Four quantities from information theory keep showing up in ML: entropy measures the average surprise in a distribution, cross-entropy is the loss that trains classifiers and language models, KL divergence measures how far one distribution sits from another, and perplexity is the intuitive branching-factor view of a language model's loss. Knowing where each appears separates people who tuned a loss function from people who only imported one.02Precision, Recall and F1Free
Precision asks how many of your positive predictions were right; recall asks how many of the real positives you caught. They trade off against each other, F1 is their harmonic mean, and accuracy lies to you the moment the classes are imbalanced.03Gradient Descent & Learning RateFree
Gradient descent is how almost every model learns: compute the slope of the loss with respect to the weights, then step the weights a little in the downhill direction. The learning rate sets the step size, and it is the single most consequential knob. Too small and training crawls; too large and it overshoots and diverges.04Bias-Variance TradeoffFree
Bias is error from a model too simple to capture the pattern; variance is error from a model so flexible it memorizes noise. Total generalization error is roughly their sum, and the whole craft of model fitting is pushing both down at once instead of trading one for the other.05Overfitting and RegularizationFree
Overfitting is when a model learns the noise in your training data instead of the signal, so it scores beautifully on data it has seen and falls apart on data it has not. You spot it from the gap between train and validation error, and you fight it with more data, regularization, early stopping, dropout, and honest cross-validation.06Golden Datasets and Eval SetsFree
A golden dataset is a representative, labeled set of examples drawn from real usage and held out from all tuning, used as the fixed yardstick for whether a change is better or worse. In classical ML it is called the test set; in LLM systems it is the eval set. Either way it is the single most valuable asset you build, because without it you are shipping on vibes.07Calibration and UncertaintyCore
A model is calibrated when its confidence matches its accuracy: of the predictions it calls 80% likely, about 80% should be right. Modern neural nets and LLMs are usually overconfident, so a raw probability or a self-reported 'I'm 95% sure' is not trustworthy on its own. You fix it with temperature scaling or isotonic regression, get distribution-free coverage with conformal prediction, and then use the calibrated confidence to abstain, route, or escalate to a human.08LLM-as-a-JudgeCore
LLM-as-a-judge uses a strong model to grade outputs against an explicit rubric, so evaluation scales past the few hundred examples a human can read by hand. It only counts as evaluation once you have calibrated the judge against 50-100 human labels and reported how well it agrees, because an uncalibrated judge is just a confident opinion.09Evaluating RAG SystemsCore
The central rule of RAG evaluation is to score retrieval and generation separately, because they fail for different reasons and you cannot fix what you cannot isolate. Retrieval is graded against a golden set with recall@k, precision@k, MRR, and NDCG; generation is graded for faithfulness and answer relevance, usually with an LLM judge. Recall@k is the ceiling on everything downstream.10A/B, Canary and Shadow TestingCore
Offline evals tell you a change is plausibly better; online testing tells you it actually is. A/B randomizes users between versions to measure real impact with statistical significance. Canary routes a small slice of live traffic to the new version to limit blast radius, and shadow mirrors real traffic to it with no user-facing effect so you can validate safely before anyone is exposed.11Multi-Armed BanditsCore
A bandit is online decision-making under uncertainty: you have several options (arms), each with an unknown payoff, and every choice both earns a reward and teaches you something. The tension is explore versus exploit: try uncertain arms to learn, or pull the current best to win. Epsilon-greedy, UCB, and Thompson sampling balance that tradeoff; contextual bandits pick per request using features. Unlike a fixed A/B test, a bandit shifts traffic toward winners as it learns, cutting the cost of running a loser.12Offline vs Online EvaluationPremium
Offline evaluation scores a change against a fixed golden set: fast, cheap, repeatable, and runnable in CI before anything ships. Online evaluation measures the change on real traffic and real users, usually via A/B, and is the only true read on impact. The two are not interchangeable: offline gains routinely fail to hold online because of distribution shift and metric gaming. The discipline FDE loops test is using offline to gate and online to confirm.13Synthetic Data GenerationCore
Synthetic data uses a strong model to manufacture training or evaluation examples when human labels are scarce or expensive: instruction/response pairs, hard edge cases, distillation targets. It works when you bolt on real quality controls (dedup, filtering, diversity, verification) and fails quietly when you skip them, because you can poison your own training set and contaminate your own benchmarks.14Benchmarks and Their LimitsCore
Public benchmarks like MMLU, HumanEval and HELM give a single comparable number, which is why they fill leaderboards. They are also a weak proxy for whether a model works on your customer's task, because of train-test contamination, overfitting to the benchmark, narrow construct validity, and the gap between a generic test and a specific job. The credible move is to build a task-specific eval set, not to quote a leaderboard.15Catastrophic ForgettingCore
When you fine-tune a model on new data, gradient updates overwrite the weights that encoded old skills, so the model gets better at the narrow new task and quietly worse at things it used to do well. It bites in practice when a model fine-tuned on a customer task loses general instruction-following. The fixes are replay data, parameter-efficient methods like LoRA, lower learning rates, regularizing toward the base, and always evaluating on a held-out general set before and after.16Loss FunctionsFree
The loss function is the objective you actually optimize, and choosing it wrong quietly sabotages everything downstream. MSE punishes outliers, MAE ignores their size, Huber splits the difference, cross-entropy is the default for classification, and contrastive losses shape embeddings. The rule: the loss must match the metric you are judged on.17Activation FunctionsFree
Without a non-linear activation, stacking layers is pointless: the whole network collapses into one linear map. Sigmoid and tanh saturate and kill gradients in deep nets, ReLU fixed that but invented dead neurons, and LeakyReLU, GELU and SiLU patch the dead-neuron problem. Softmax is for outputs, not hidden layers.18Normalization: Batch vs LayerCore
Normalizing activations keeps their scale stable as they flow through a deep net, which speeds training and lets you use higher learning rates. Batch norm normalizes across the batch and has nasty train/eval and small-batch pitfalls; layer norm normalizes across features per example, which is why transformers use it. RMSNorm strips it down further.19Handling Imbalanced DataCore
On a 99-to-1 class split, a model that predicts the majority class scores 99% accuracy and catches nothing, which is why accuracy lies on imbalanced data. The fixes are resampling (SMOTE, undersampling), class weights, and threshold moving, judged on PR-AUC and recall-at-precision, not accuracy. The biggest pitfall is resampling before you split.20Neural Network Basics: Perceptron to MLPFree
A perceptron is a single linear unit with a threshold: it can only separate data a straight line can split, which is why it famously cannot learn XOR. Stack hidden layers with a non-linear activation and you get a multilayer perceptron, a universal approximator that learns its own features. The forward pass produces predictions; backprop is just gradient descent through the chain rule.21Semi-Supervised and Self-TrainingCore
Semi-supervised learning uses a small labeled set plus a large unlabeled pool. Self-training labels the unlabeled data with the model's own confident predictions and retrains; consistency regularization forces the model to give the same answer to perturbed copies of an input. It helps when labels are scarce but unlabeled data is plentiful and the cluster assumption holds, and it backfires through confirmation bias when the model is wrong but confident.22Convex vs Non-Convex OptimizationCore
A convex loss has one global minimum, so gradient descent from anywhere reaches the best solution: this is why logistic regression, linear regression, and linear SVMs are reliable to train. Deep nets are non-convex, with many local minima and far more saddle points, yet SGD still finds good solutions. Understanding why changes how you set initialization, learning rate, and restarts.23Computer Vision: Classification, Detection, SegmentationCore
Vision tasks form a ladder of increasing spatial precision: classification labels the whole image, detection draws bounding boxes around objects, semantic segmentation labels every pixel by class, and instance segmentation separates individual objects pixel by pixel. Each has its own output format and metric (top-1 accuracy, mAP at IoU thresholds), and CNNs versus vision transformers trade off differently across them.04
⚙️ System Design for AI in Production
Turning a notebook demo into a deployment customers trust: idempotency, retries, observability, latency, and private deploys.
01From Proof-of-Concept to ProductionFree
A notebook that answered one question correctly during a demo is not a deliverable. Production is the unglamorous work that turns a one-time success into a system the customer can run, trust, and operate without you in the room. Closing that gap is most of the FDE job.02AI Cost and Unit EconomicsCore
Unit economics is the napkin math that decides whether an AI deployment ships: cost per request driven by input and output tokens, multiplied by volume, against the human or manual baseline it replaces. It also governs the API-versus-self-host break-even, which only flips in favor of your own GPUs above a real utilization threshold.03Retries, Exponential Backoff and JitterFree
When a call fails on a transient error, retrying immediately is the worst thing you can do: it piles load onto an already-struggling service and synchronizes every client into a stampede. Exponential backoff spaces retries out, and jitter de-synchronizes the clients so they stop arriving in lockstep.04IdempotencyCore
An idempotent operation produces the same end state whether you apply it once or five times, which is exactly what you need in a world where retries and at-least-once delivery mean every request may arrive twice. Without it, a single lost response turns one charge into two; with it, the duplicate is a no-op.05Rate LimitingCore
Rate limiting caps how fast a client or your whole fleet can hit a resource, so a burst of traffic or one runaway caller cannot melt a fragile downstream service or burn your third-party API quota. The interesting part is enforcing it across many machines without a race, and deciding whether to reject or queue when the limit hits.06Observability for AI SystemsCore
You cannot operate what you cannot see, and an AI system has failure modes a normal service does not: the prompt, the retrieved context, the model output, and the slow drift in quality over time. Observability for AI means logging and tracing every stage of the chain with a shared request ID, so when an answer is wrong you can reconstruct exactly why.07Latency OptimizationCore
Measure p50, p95, and p99 before you touch anything, then find where the time actually goes: tokenization, retrieval, inference, or post-processing. A naive RAG pipeline that takes 1.5 seconds can usually reach sub-100ms perceived latency by caching, parallelizing retrieval, picking a smaller model, and streaming the first token, in that order of payoff.08VPC and Air-Gapped DeploymentPremium
Large enterprises will not let their data leave their security boundary, so you deploy your software inside the customer's private VPC, reach their data over PrivateLink instead of the public internet, authenticate through their SSO, and encrypt everything at rest and in transit. For truly air-gapped environments you ship self-hosted models too. This is core Forward Deployed Engineer work.09Circuit Breakers and BackpressurePremium
A circuit breaker stops calling a dependency that is already failing, so one sick service does not drag down everything that depends on it. Backpressure is the upstream half of the same fight: when a downstream cannot keep up, you signal callers to slow down or shed load instead of piling work into an unbounded queue.05
🔁 MLOps & Lifecycle
Shipping and operating models safely: drift, model registries, CI/CD, monitoring, and feature stores.
01Data and Concept DriftCore
A model can lose accuracy two ways: the inputs it sees start looking different (data drift), or the true mapping from inputs to outputs changes underneath it (concept drift). The fix differs, so the FDE skill is diagnosing which one you have before reaching for a retrain.02Model Registry and PromotionCore
A model registry is the source of truth for every model version, what data and code produced it, and how it scored on your eval suite. Promotion is the gated path from a registered candidate to live serving: pass the gates, soak in shadow or canary, then swap an alias so traffic moves atomically and rollback is one step.03CI/CD for ModelsCore
Model CI/CD looks like code CI/CD but ships data, weights, and prompts together, and its merge gate is an eval suite against a golden set, not a passing unit test. The pipeline trains, evaluates, registers, soaks in shadow or canary, then promotes, with every input versioned so any release is reproducible.04Model MonitoringCore
Model monitoring is watching a deployed model's health the way you watch a service: prediction distributions, input drift, latency, error and abstain rates, and the business metric the model is supposed to move. The skill interviewers test is triage: telling a model problem apart from a data or pipeline problem, and knowing which signal fires first.05Feature StoresPremium
A feature store is a central place that computes a feature once and serves it to both training (offline, batch) and serving (online, low-latency) from the same definition, which kills the most common production bug in ML: train/serve skew. It also handles point-in-time correctness so backfills do not leak the future. The honest catch is that most early-stage teams do not need one.06
🖥️ ML Infrastructure & Serving
Where the GPUs live: memory, quantization, high-throughput serving, and the tricks that make inference cheap and fast.
01GPU Memory and VRAMCore
VRAM is the budget that decides which models you can actually run. It is spent on three things: model weights, the KV cache, and activations. Knowing the back-of-envelope arithmetic (a 7B model at fp16 is roughly 14GB of weights) is what separates a candidate who has deployed an LLM from one who has only read about it.02QuantizationCore
Quantization stores model weights (and sometimes activations) in fewer bits, fp16 down to int8 or 4-bit, which cuts memory and speeds inference. The quality hit is usually small at int8 and larger at 4-bit. Knowing post-training quantization versus quantization-aware training, and when each is acceptable, is standard FDE interview ground.03Knowledge DistillationCore
Distillation trains a small student model to mimic a large teacher, learning from the teacher's full output distribution rather than just hard labels. The soft targets carry extra signal about how the teacher 'thinks', so the student keeps much of the quality at a fraction of the size and latency. Knowing when distillation beats quantization or pruning is standard FDE ground when you have a latency or cost budget to hit.04Inference Serving (vLLM, TGI)Core
Serving LLMs at high throughput under a latency SLO is its own engineering problem. Continuous batching keeps the GPU busy across requests of different lengths, and PagedAttention stops the KV cache from wasting memory. Naive one-request-at-a-time serving leaves most of an expensive GPU idle, which is why purpose-built runtimes like vLLM and TGI exist.05Continuous BatchingPremium
Static batching runs a fixed group of requests to completion together, so a batch of one short reply and one long reply makes the GPU idle while it waits on the longest. Continuous batching adds and evicts sequences from the running batch every decode step, keeping the GPU saturated and multiplying throughput. It is the scheduling trick at the heart of vLLM and every modern LLM serving stack.06PagedAttentionPremium
Allocating each sequence's KV cache as one contiguous block forces you to reserve space for the maximum possible length, which fragments GPU memory and wastes most of it. PagedAttention borrows OS-style paging: it stores the KV cache in fixed-size non-contiguous blocks tracked by a per-sequence block table, so memory is packed, grown on demand, and shareable across sequences. It is the core memory trick that lets vLLM pack far more concurrent sequences into the same VRAM.07Speculative DecodingPremium
Decoding is slow because each token needs a full forward pass through a huge memory-bound model. Speculative decoding has a small fast draft model propose several tokens at once, then the large target model verifies them all in a single forward pass and keeps the longest correct prefix. A careful accept rule makes the output provably identical to sampling from the target model, so you get lower latency for free, not an approximation.08Distributed Training (FSDP, Parallelism)Premium
When a model or its training state will not fit on one GPU, you split the work across many. Data parallelism replicates the model and splits the batch; tensor parallelism splits a single layer's math across GPUs; pipeline parallelism splits the layer stack into stages; and FSDP/ZeRO shard the parameters, gradients, and optimizer states themselves. Each buys memory by spending network bandwidth, so the real skill is composing them to fit the model while keeping the GPUs busy.09GPU Architecture and ExecutionCore
A GPU is not a fast CPU. It runs thousands of threads in lockstep groups called warps across many streaming multiprocessors, under the SIMT model, and its real constraint is moving data through a memory hierarchy that spans fast on-chip registers and shared memory down to slow off-chip HBM. Understanding occupancy, coalesced memory access, and warp divergence is what separates a kernel that hits peak throughput from one that leaves 90% of the chip idle.07
🗄️ Data & SQL Engineering
The data plumbing under every AI deployment: window functions, idempotent pipelines, data quality, and change capture.
01SQL Window FunctionsFree
Window functions compute a value across a set of rows related to the current row without collapsing them, so you can rank, compare to a neighbor, or run a cumulative total while keeping every row. They are how analysts answer 'compared to what?' questions in pure SQL, and most interviewers use them to tell people who know SQL from people who know GROUP BY.02Idempotent Data PipelinesCore
Pipelines retry, get re-run, and get backfilled, and every one of those re-runs must produce the same result as running once. Idempotency is the property that makes that true: write by key with upsert or partition overwrite, never blind append, so a retry cannot double-count. It is the single property that makes a pipeline safe to operate, because the alternative is a 2 a.m. page where you cannot tell if it is safe to run the job again.03Data Quality and ValidationCore
A deployment lives or dies on the customer's data, and that data is worse than their sample suggested. The job is to build automated quality gates (schema, null, range, uniqueness, freshness) at the boundary, quarantine bad records instead of failing the whole batch, and alert on the rate so a Tuesday-shaped degradation surfaces before a dashboard goes wrong. This is the difference between a pipeline that fails loudly and one that lies quietly.04Deduplication and LSHCore
Exact duplicates fall out of hashing the normalized content, but near-duplicates (the same record with a typo, a reordered address, boilerplate that repeats across documents) need similarity, not equality. MinHash estimates Jaccard similarity cheaply, and Locality-Sensitive Hashing buckets similar signatures so you only compare likely pairs instead of all O(n^2). This is a constant reflex when merging messy enterprise data and when curating training corpora.05Gaps and IslandsPremium
Gaps and islands is the SQL pattern for collapsing a sequence of rows into the contiguous runs (islands) and the breaks between them (gaps). The trick is a difference of two row numbers that stays constant inside a run, giving every row in the same island an identical group key you can then aggregate. It powers sessionization, login streaks, and contiguous date-range queries, and interviewers love it because the naive self-join answer is both slow and wrong on ties.06Change Data Capture (CDC)Premium
Change Data Capture streams row-level inserts, updates, and deletes out of a source database so downstream systems stay in near-real-time sync without full reloads. The strong form reads the database transaction log rather than polling tables, which captures deletes, preserves commit order, and adds almost no load to the source. The hard parts are ordering, tombstones for deletes, and applying the stream idempotently so a replay does not corrupt the target.08
🛡️ AI Security, Privacy & Governance
Keeping enterprise deployments safe and compliant: prompt injection, PII, tenant isolation, audit trails, and governance regimes.
01Prompt Injection and DefenseCore
Prompt injection is the attack where untrusted text smuggles instructions into a model's context and overrides the system's intent. It comes in two flavors: direct, where the user types the attack, and indirect, where a poisoned document or tool output the model later reads carries it. You cannot fully prevent it, so a competent FDE designs the system so that a successful injection cannot reach anything that matters.02PII Handling and RedactionCore
Personal data leaks into AI systems through three doors: the prompt you send a model API, the logs you keep for debugging, and the traces you store for evaluation. Handling it means detecting and redacting personal data before it crosses any of those boundaries, then minimizing, encrypting, access-controlling, and expiring whatever you must keep. In regulated industries, logging a raw prompt is the single most common compliance failure.03Differential PrivacyCore
Differential privacy is a mathematical guarantee that the output of a computation barely changes whether or not any single person's record was included, so an attacker studying the output cannot confidently tell who was in the data. You buy this guarantee by adding calibrated random noise, and you pay for it in accuracy. The privacy budget epsilon sets the exchange rate; smaller epsilon means more noise and more privacy, and a value like epsilon = 8 is moderate, not strong.04Audit Trails and TraceabilityCore
An audit trail is an immutable, queryable record that lets you reconstruct, months later, exactly who triggered a given AI output, with which model and prompt version, over which data. It is evidence for a regulator or a customer's security team, not a debugging log, and the difference is design: correlation IDs threaded end to end, tamper-evidence, and a retention policy. Regulated buyers will not sign without it.05Federated LearningCore
Federated learning trains one shared model across many devices or organizations without moving their raw data to a central server. Each participant trains locally on its own data and sends back only model updates, which a server averages into a new global model. It is the pattern an FDE reaches for when data legally or physically cannot leave its owner: hospitals, banks, and phone keyboards. The catch is that raw updates can still leak information, so real deployments layer on differential privacy or secure aggregation.06Multi-Tenancy and Data IsolationPremium
Multi-tenancy is serving many customers from shared infrastructure while guaranteeing no tenant can ever see another's data. The isolation strategies run a spectrum from row-level filtering to fully separate databases, trading cost against blast radius. The non-negotiable rule for AI systems: tenant scope is enforced below the model, in code that filters queries and scopes credentials, never by instructing the model in a prompt. A single prompt-injected document is enough to break prompt-level isolation.07Mechanistic InterpretabilityPremium
Mechanistic interpretability tries to reverse-engineer the actual computations inside a model rather than treating it as a black box: finding the features it represents and the circuits that combine them. The current toolkit centers on sparse autoencoders that decompose dense activations into interpretable features, causal tests like activation patching that prove a feature matters, and steering that turns a behavior up or down at inference. Be honest in interviews: nobody can fully explain a frontier model, you cannot prove a behavior is absent, and feature labels are human guesses.08AI Governance (SOC2, EU AI Act)Premium
These are the regimes an enterprise FDE actually meets in the field: SOC 2 for security and availability controls with documented evidence, the EU AI Act for risk-tiered obligations on high-risk AI, and data-protection law like GDPR and India's DPDP. In practice they all demand the same primitives: documented controls, human oversight of consequential decisions, audit trails, and disciplined data handling. The FDE move is to design for them from the first deployment, because retrofitting governance into a shipped system is far more expensive than building it in.09
💻 Coding & Engineering Craft
The practical engineering FDE screens reward: parsing messy data, testable design, streaming, and the Big-O that genuinely matters.
01Parsing Messy, Real-World DataFree
Customer files are dirty: inconsistent quoting, missing headers, junk rows, encodings that lie. The job is to parse defensively, skip and log bad rows instead of aborting the whole batch, and keep parsing pure and separate from business logic so it stays testable and deterministic. This is most of what early FDE data-ingestion work actually is.02Big-O That Actually MattersFree
On a deployment, Big-O is not a whiteboard puzzle; it is the one calculation that tells you whether the customer's data fits in the approach you picked. The skill is spotting the term that dominates at their scale, knowing when brute force dies and you need an index or ANN, and recognizing when constant factors and memory decide the outcome instead of the exponent.03Testability and Dependency InjectionCore
Code that reaches out to the clock, the network, the filesystem, or a random generator cannot be tested deterministically, because its output depends on the world. The fix is to separate pure logic from side effects and inject the things that touch the world (the clock, I/O, randomness) so a test can pass fakes. When you inherit untestable code, pin its current behavior with a characterization test first, then refactor under that net.04Streaming and BackpressureCore
Streaming processes data one chunk at a time so memory stays flat no matter how big the input is. The moment a producer outruns its consumer, you need backpressure: a bounded buffer that makes the producer wait instead of piling unbounded work into memory. In Python this is generators and chunked reads for the streaming half, and a bounded queue (or a blocking put) for the backpressure half. Get it wrong and a 50 GB file or a fast upstream OOMs the box.05Sliding Window and Two PointersFree
A huge fraction of array and string screens are really one of two patterns. Two pointers walk a sorted structure from both ends or at two speeds; the sliding window keeps a running answer over a contiguous range and slides instead of recomputing. Both turn an obvious O(n^2) double loop into a single O(n) pass, and recognizing which one applies is most of the battle in a 25-minute screen.06Heaps and Top-KFree
When a problem says top-K, K-th largest, or merge K sorted streams, the answer is almost always a heap. A binary heap gives you the smallest (or largest) element in O(1) and insert/pop in O(log n), which turns a full O(n log n) sort into an O(n log k) scan when you only need the K best. The recurring trick, counterintuitive at first, is to keep a min-heap of size K to find the K largest.07Graph Traversal and Topological SortCore
Grids, dependency chains, task schedulers, and path problems are all graphs in disguise. BFS finds shortest paths in unweighted graphs and explores level by level; DFS goes deep and is the backbone of cycle detection. Topological sort orders a DAG so every dependency comes before what needs it, and the same machinery tells you whether a dependency graph has an impossible cycle.08Caching and EvictionCore
A cache is bounded memory in front of expensive work, so the real design question is what to throw away when it fills. LRU evicts the least recently used entry and is the default; it is built from a hash map plus a doubly linked list to get O(1) get and put. TTL adds time-based expiry. Picking and implementing the right eviction policy is one of the most common practical FDE coding screens.09Concurrency and the GILCore
Concurrency screens punish people who reach for threads without knowing what Python's GIL does. The GIL means threads do not run Python bytecode in parallel, so threads help I/O-bound work and do nothing for CPU-bound work, which needs processes. The other half is correctness: shared mutable state needs a lock, and a bounded queue is the clean way to hand work between producers and consumers without races.10Numerical StabilityCore
When you implement softmax, cross-entropy, or a running average by hand, the naive formula overflows or loses precision on real inputs. The fixes are a small, reusable toolkit: subtract the max before exponentiating, work in log space with log-sum-exp, and accumulate carefully. ML-adjacent coding screens lean on this, because the candidate who writes exp of a large logit and gets inf has shipped a silent bug.10
🤝 The Customer-Facing Craft
The half of the job most engineers under-train: discovery, scoping ambiguity, translating trade-offs, and handling the room.
01Requirements DiscoveryFree
Requirements discovery is the work of finding the real problem hiding behind the customer's stated ask. The request they hand you ("build us a chatbot") is almost never the need; the FDE who surfaces who uses it, what success looks like, what data actually exists, and why the deadline is the deadline is the one who ships something people use.02Scoping Ambiguous ProblemsFree
Scoping an open-ended prompt ("a city wants to reduce 911 response times") is a structured move, not a flash of inspiration: clarify inputs and constraints, state your assumptions out loud, carve out the smallest useful MVP, name the accuracy/cost/latency trade-offs you are choosing, and plan for what happens when it fails. Diving straight into a model or an architecture is the most common reason candidates get cut in the simulation round.03Explaining Trade-offs to Non-EngineersFree
An exec does not care whether you chose RAG or fine-tuning; they care what it costs, when it ships, and what it might get wrong. Translating a technical trade-off means converting accuracy, cost, and latency into the decision the business is actually making, framing each option as a choice with a consequence in their terms, and answering the question they will all eventually ask: why does the AI give a different answer every time, and why is that not a bug.04Stakeholder ManagementCore
A deployment spans the analyst who will use the tool daily and the CTO who signed the check, and those people want different things. Stakeholder management is figuring out who actually decides, building enough trust to be believed when you deliver bad news, and managing expectations so reality never arrives as a surprise. The job is not shipping the system; it is getting people to adopt it, which is a different and harder thing.05Recovering a Failing Live DemoCore
Mid-demo the system throws a stack trace on the projector in front of the customer's executive team. The recovery is not technical heroics; it is composure plus parallelism: keep talking and orienting the room while a teammate triages off-screen, fall back to a prepared path before the silence spikes anxiety, and never blame the data, a teammate, or the room. A clean recovery routinely builds more trust than a flawless demo, because executives are watching how you behave when the thing breaks.