Study Guides/AI-300/Optimize Generative AI Systems and Model Performance

10-15% of exam

Optimize Generative AI Systems and Model Performance

Tune RAG retrieval quality with chunking and hybrid search, and apply advanced fine-tuning and synthetic data to customize model behavior.

Design and Implement an MLOps Infrastructure Implement Machine Learning Model Lifecycle and Operations Design and Implement a GenAIOps Infrastructure Implement Generative AI Quality Assurance and Observability Optimize Generative AI Systems and Model Performance

RAG optimization: chunking, embeddings, and similarity thresholds

Retrieval quality sets a ceiling on generation quality — no amount of prompt engineering fixes a response built from the wrong retrieved chunks, which is why RAG optimization starts at indexing time, not at the model call.

Chunk size and overlap

Chunking exists partly to respect embedding and chat model token limits (for example, `text-embedding-3-small`'s roughly 8,191-token input limit), and partly because retrieval precision degrades when a chunk mixes multiple unrelated topics together.
Azure AI Search's Text Split skill chunks by `pages` (character-based) or `sentences`, controlled by `maximumPageLength` and `pageOverlapLength` — a commonly cited reasonable default is a 2,000-character page length with 500 characters of overlap, though the right values depend on document structure and how the same chunks will be used (a chunk size that works well for embedding might not be ideal for summarization if both share the same pipeline).
Overlap exists specifically to avoid losing context that straddles a chunk boundary — too little overlap can silently split a sentence's meaning across chunks that then can't be independently understood if only one is retrieved.

Embedding model selection

The embedding model determines both retrieval quality and cost/latency per indexed document and per query — a larger embedding model generally captures finer semantic distinctions but costs more to run against every chunk at index time and every query at retrieval time.
Embedding model choice and chunk size interact: a model with a small context window forces smaller chunks regardless of what would otherwise be an ideal chunk size for the content itself.

Similarity thresholds and hybrid search

A vector query alone can return "the closest matches available" even when none of them are actually a good semantic match for the query — a minimum similarity threshold filters out results below a relevance cutoff rather than always returning the top-K regardless of quality.
Hybrid search runs a keyword (full-text) query and a vector query in parallel over the same request, then merges and reorders results using Reciprocal Rank Fusion (RRF) — this offsets each approach's weakness individually: keyword search misses paraphrased/semantically-similar-but-differently-worded content, while vector search alone can miss exact terminology or identifiers (product codes, names) that a keyword match would catch precisely.
Semantic ranking adds a second-stage rescoring pass on top of hybrid search's initial results, using a more sophisticated model against just the top candidates — a two-stage design because a heavier, more accurate ranking model would be too costly to run against every document in a large index directly.

Common confusion

Increasing chunk size doesn't uniformly improve retrieval — larger chunks preserve more context per retrieved unit (good for narrative or explanation-heavy content) but reduce retrieval precision (a large chunk containing the answer buried among unrelated text scores as "relevant" even though most of it isn't), so chunk sizing is a genuine trade-off tuned per content type and use case, not a setting to maximize.

Measuring and testing retrieval relevance

Tuning any of the RAG knobs above without a way to measure whether a change actually helped is just guessing — relevance measurement and structured comparison are what turn RAG optimization into an evidence-based process.

Relevance metrics

`RetrievalEvaluator` and `DocumentRetrievalEvaluator` (from the same evaluation SDK used for generation quality) score the retrieval step specifically — whether the system fetched relevant material at all — separately from whether the final generated answer used that material well.
Standard information-retrieval metrics (precision/recall over labeled relevant documents, and rank-aware metrics that reward relevant results appearing near the top) apply directly to a RAG retriever's output, since retrieval is fundamentally a search-ranking problem wearing a generative-AI hat.

A/B testing retrieval and prompt changes

Because a RAG pipeline has multiple independently tunable stages (chunking strategy, embedding model, similarity threshold, hybrid vs. vector-only, reranking on or off, and the generation prompt itself), isolating which change actually caused an improvement requires changing one variable at a time against a fixed, representative evaluation dataset — not the entire pipeline in one attempt.
A/B testing in production (splitting live traffic between a current and candidate configuration and comparing evaluation metrics, user engagement, or task completion) is the natural extension of offline evaluation once a candidate configuration looks promising in testing but its real-world impact still needs confirmation under live conditions.

Metadata filtering alongside similarity ranking

Combining a vector similarity `ORDER BY`-style ranking with a metadata `WHERE`-style filter (tenant ID, document category, date range) narrows the search space to only documents a given user or use case should ever see, evaluated in the same query as the similarity ranking rather than as a separate post-filter step.
Getting the filter/ranking order wrong (filtering after retrieving only the top-K by similarity, instead of filtering the candidate set before or during ranking) can silently return fewer results than expected, or worse, leak content that should have been excluded if the filter is applied only cosmetically to the response rather than to the actual search.

Common confusion

A high similarity score between a query and a retrieved chunk doesn't guarantee the chunk is actually useful for answering that query — semantic similarity measures topical closeness, not whether the specific fact or instruction the user needs is actually present in that chunk, which is exactly the gap groundedness and relevance evaluation (applied to the final generated response) are designed to catch that a retrieval-only similarity score can't.

Advanced fine-tuning methods

Fine-tuning changes a model's weights based on examples, rather than changing what's included in each prompt at inference time — the right tool when consistent behavior needs to be baked in rather than re-specified in every call.

Training methods

Supervised fine-tuning (SFT) is supported by essentially all non-reasoning models and trains on labeled input/output examples directly — it's the default starting point for teaching a model a specific format, tone, or domain-specific behavior.
Direct Preference Optimization (DPO) trains from *pairs* of preferred vs. non-preferred responses to the same input, rather than a single "correct" output — useful when the goal is steering style or judgment calls where there isn't one unambiguous right answer, only a comparative preference.
Reinforcement Fine-Tuning (RFT) is for reasoning models and grades the model's output against a scoring function (a grader) rather than an exact reference answer — training data omits system messages, the final message must come from the user, and datasets can be considerably smaller (dozens to a few hundred examples to start) than SFT typically needs, though every training file goes through an automated safety screening pass before training can begin.

Data preparation

Training and validation data are uploaded as JSONL files (UTF-8, under the size limit for the direct upload API — larger files use a separate multipart uploads API), with each line's schema differing by method: SFT and DPO follow a chat-messages format, RFT's format adds fields a grader needs and drops system messages entirely.
Training and validation data must not overlap — validation exists specifically to catch overfitting, which reusing training examples for validation would silently hide.

Synthetic data for fine-tuning

Where real labeled examples are scarce (a new domain, a rare intent, an edge case that hasn't occurred in production yet), synthetically generating example input/output pairs (often using a stronger model to generate candidate examples reviewed for quality) fills the gap — the same "combine production and synthetic data" pattern used for evaluation datasets applies to training data too.
Synthetic data quality matters more as volume increases — a larger volume of *low-quality* synthetic examples can actively degrade a fine-tuned model's behavior rather than improve it, since the model has no way to distinguish a synthetic example's subtle errors from a deliberate lesson.

Common confusion

Fine-tuning and RAG solve different problems and are often confused as interchangeable "make the model know more" techniques: fine-tuning changes how the model behaves (format, tone, task-specific judgment) based on training examples baked into its weights, while RAG supplies current, specific facts at inference time without retraining anything — a model fine-tuned on last quarter's documents still won't know about a document added yesterday, which only retrieval can supply.

Monitoring and lifecycle management of fine-tuned models

A fine-tuned model is a distinct, deployed artifact with its own cost, performance profile, and drift risk — its lifecycle doesn't end once training completes, and treating it as "set once and done" is a common way production quality regresses silently.

From training to deployment

A completed fine-tuning job produces a fine-tuned model identifier that then has to be explicitly deployed, exactly like a base model — Global-tier training (cheaper, faster queuing, but copies data/weights outside the resource's region) versus Standard-tier training (keeps data in-region, for data-residency requirements) is chosen at job creation, not at deployment time.
Fine-tuned model deployments bill hourly for hosting *in addition to* training-time and inference-time token costs — an easy way to accumulate unnecessary spend is forgetting to delete a fine-tuned deployment that's no longer being evaluated or used, since the hosting cost accrues whether or not it's actively receiving traffic.

Monitoring a fine-tuned model in production

The same evaluation and observability practices covered earlier (built-in evaluators, token/latency/cost tracking, tracing) apply to a fine-tuned deployment exactly as they do to a base model deployment — fine-tuning doesn't exempt a model from ongoing quality or safety evaluation.
Fine-tuned models carry an additional drift risk specific to them: as the underlying task or domain evolves, a model fine-tuned on last year's examples can degrade in ways a base model wouldn't, since its specialized behavior was learned from a specific, dated snapshot of examples rather than kept current by retrieval.

Full lifecycle management

Retraining a fine-tuned model (with updated or expanded training data) follows the same develop → evaluate → compare-against-current-production → promote pattern as any other model lifecycle change — the new fine-tuned version is evaluated against the same test set the current production version was validated against, not just checked in isolation.
Versioning fine-tuned models the same way as any other deployed model (keeping the previous fine-tuned deployment available, shifting traffic gradually, and being able to roll back to it) applies the safe-rollout practices from the model lifecycle domain to fine-tuned models specifically, rather than treating a fine-tuned model swap as a special, riskier, all-or-nothing cutover.

Common confusion

Successfully completing a fine-tuning job and seeing good training/validation metrics doesn't mean the resulting model is ready for production traffic — those metrics reflect performance against the specific training and validation data used, not against the full range of real-world inputs the evaluation suite (groundedness, relevance, safety) is designed to catch problems across.

Back to AI-300 Overview Practice Questions