AI-103 Azure AI Search: Retrieval Pipelines and Document Extraction

Introduction

This lesson covers the Implement information extraction solutions domain of AI-103 (10–15%). The official skills are in two groups. Retrieval and grounding pipelines: ingesting and indexing documents, images, audio and video; semantic, hybrid and vector search; enrichment skills; and RAG ingestion with OCR. Extracting content from documents: multimodal pipelines that combine OCR, layout analysis and field extraction, and producing structured or Markdown output with Azure Content Understanding in Foundry Tools. Your course outline also includes connecting retrieval to agent tools.

Building on generative applications and RAG with Foundry, Azure AI Search retrieval pipelines connect extracted evidence to grounded answers. This lesson goes upstream: how the knowledge gets into the index in good shape, how it is searched, and how structured data is pulled out of messy files.

After this lesson you will be able to design an ingestion pipeline for mixed content, choose the right search mode, configure enrichment and OCR, choose between Document Intelligence-style layout extraction and Content Understanding analyzers, and expose retrieval to an agent as a tool.

Learning objectives

• Design ingestion for documents, images, audio and video, including chunking and metadata.
• Choose keyword, vector, hybrid and semantic search, and explain why hybrid plus semantic ranking is the usual default.
• Configure an Azure AI Search indexer with a skillset (OCR, layout, split, embed) and integrated vectorization.
• Use Content Understanding analyzers to produce Markdown and structured fields, with confidence and grounding.
• Keep the index fresh, secure and measurable.
• Connect retrieval to an agent through a search tool or knowledge source.

1. The shape of a retrieval pipeline

Sources → Ingest → Extract/Enrich → Chunk → Embed → Index → Query → Rank → Ground the model
 (blobs,   (indexer  (OCR, layout,   (split   (vector   (Azure AI   (keyword  (semantic   (cited
  SharePoint, or code) transcripts,   with     model)    Search)     +vector)  ranker)     answer)
  databases)         descriptions)   overlap)

Quality is decided mostly before a query is ever asked. A model cannot cite what was lost during extraction: a table flattened into scrambled text, a scanned page that was never OCR'd, or a chunk cut in the middle of a clause.

1.1 Content types and what to extract

2. Azure AI Search as the retrieval engine

Azure AI Search provides an index that holds text fields, filterable metadata and vector fields, plus several ways to query it.

2.1 Search modes

Default recommendation: hybrid retrieval (keyword + vector) with the semantic ranker, using filters for security and metadata. This is the combination Microsoft's guidance recommends for most RAG workloads. Treat it as an evaluation baseline, not a universal answer: exact-match requirements, latency, pricing and relevance measurements determine the final configuration.

2.2 Vector search details

• An embedding model (for example a text-embedding deployment) converts text, and for multimodal models also images, into vectors. Query and document vectors must use compatible versions of the same embedding model and dimensions, with consistent preprocessing. Changing the embedding model means re-indexing.
• Vector fields use HNSW (approximate nearest neighbour) or exhaustive KNN. HNSW is the default for scale; exhaustive search is exact and suits small sets or tests.
• Vector compression (scalar or binary quantization, and dimension truncation where the model supports it) reduces memory and cost with a small recall trade-off; oversampling and rescoring recover quality.
• Vector queries can be run in parallel to several fields (for example a text vector and an image vector) and combined with keyword queries.

2.3 Filters, security and scoring

• Filters (OData) narrow results by metadata such as department eq 'HR' or language eq 'fr'. For vector queries, select preFilter where appropriate to search within eligible candidates. Filtering strategy affects recall; enforce authorization regardless of ranking mode.
• Security trimming: store access-control identifiers (group IDs) in a filterable field and filter on the caller's groups; Azure AI Search also supports document-level access control integration with Microsoft Entra ID for some sources. Derive permissions from the authenticated caller in trusted backend code, never from model-supplied group names. Native ACL integrations vary by source, API and preview status. Never rely on the model to hide documents.
• Scoring profiles can boost freshness or fields. Use them after measuring.

Worked example 1: choosing the search configuration

Scenario. Support engineers search a knowledge base for questions such as "Error 0x80070057 when installing agent v4.2" and "how do I recover a locked device". The corpus has 600,000 chunks. Engineers complain that paraphrased questions miss articles and that error-code queries return vague results.

Reasoning.

1. Error codes and version numbers need exact tokens → keyword search.
2. "Recover a locked device" may be written as "unlock a blocked terminal" in articles → vector search.
3. Therefore run hybrid queries so both signals contribute, merged by RRF.
4. Add the semantic ranker, which reranks up to 50 candidate results; use k = 50 for the vector leg when supplying hybrid candidates, then return a smaller evidence set to the model.
5. Add a product filter derived from the user's context.
6. Measure with a test set of 200 real questions: recall@5 and the share of answers where the cited chunk was correct.

Why not vector-only? It loses exact-token precision. Why not keyword-only? It misses paraphrase. Why not just a bigger model? The failure is at retrieval, not generation.

3. Ingestion: indexers, skillsets and integrated vectorization

3.1 Indexers

An indexer pulls from a supported source such as Blob Storage, ADLS Gen2, Azure SQL or Cosmos DB; source support and preview status differ. Configure schedules, supported change tracking and deletion detection explicitly. For Blob Storage, new/changed blobs are tracked automatically, but deletion needs a supported soft-delete policy established before the first run. Physical deletion alone can leave orphan chunks. In push pipelines, application code owns changes and deletes.

3.2 Skillsets (enrichment)

A skillset is a pipeline of skills that run on each document between extraction and indexing. Built-in skills include:

For one output document, use output field mappings. For one source producing many chunks, use index projections with a parent key, chunk key and inherited security/provenance fields; mapping all chunks into one scalar field loses retrieval units. Supply an Azure AI services / Foundry resource for billing of built-in skills beyond the free tier, and prefer managed identity over keys.

An important distinction: a knowledge store can project enriched data to Blob or tables for analytics, but the search index is what serves retrieval.

3.3 Integrated vectorization

Integrated vectorization lets the indexer chunk and embed documents during indexing, and lets the index embed query text at query time with a vectorizer. Configure the index-time embedding skill and query-time vectorizer separately with matching model, version and dimensions; clients can then send text-kind vector queries. It reduces custom code and keeps document and query embeddings consistent. Pushing precomputed document vectors does not prevent use of a matching query vectorizer; otherwise, embed queries in application code.

3.4 Chunking

Each chunk should carry metadata: source title, URL or path, page number, section heading, date, language and access groups. Metadata powers filters, citations and freshness.

3.5 Keeping the index healthy

• Schedule indexer runs and use change tracking; monitor failures and skipped documents.
• Re-run enrichment when you change a skill (skillset versions) using the reset/reprocess feature.
• Track search latency, throttling, index size, and relevance metrics; evaluate after every change to chunking, embeddings or ranking.
• Plan capacity with replicas (query throughput and availability) and partitions (storage and indexing).

Worked example 2: RAG ingestion for scanned contracts

Scenario. A law firm has 80,000 scanned PDF contracts in Blob Storage. Lawyers ask questions like "Which agreements allow termination for convenience with 30 days' notice?" Answers must cite the page. Some pages are rotated or have tables of fees.

Reasoning.

1. Create an indexer over the container with change tracking and a supported deletion policy before the first run.
2. In the skillset, use Document Layout to extract OCR text, structure and page/location metadata, then split on headings with token limits. Alternatively evaluate the built-in Content Understanding skill; its GA document parsing differs from opt-in preview semantic chunking and image descriptions. Both skills have processing timeouts, so preprocess oversized files.
3. Preserve the heading with each clause, split oversized sections at coherent boundaries and verify table/footnote continuity; headings alone do not guarantee safe chunk boundaries.
4. Embed chunks and use index projections to create per-chunk records with text, vector, page, contractId, client, effectiveDate and groups.
5. Query with hybrid + semantic ranker, filter on groups for security, and return chunk text plus page numbers for citation.
6. Add an LLM-extracted field termination_notice_days during indexing (via a custom skill or Content Understanding) so numeric questions can use a filter instead of relying on vector similarity.

Why OCR matters. Without it, scanned pages produce empty text and are invisible to search. Why structure-aware chunking? A fixed 500-token cut can separate a notice period from the clause heading, producing wrong answers.

4. Azure Content Understanding for documents and mixed content

Azure Content Understanding in Foundry Tools converts unstructured content into structured, application-ready output. It combines content extraction (OCR, layout, transcription) with generative field extraction, and covers documents, images, audio and video.

4.1 Output types

4.2 Analyzer types

As checked on 27 October 2026, use Content Understanding API 2025-11-01 for GA workloads; 2026-06-01-preview adds document agentic mode and other opt-in features. Treat preview features as subject to change, and prefer GA for production.

4.3 A custom analyzer in outline

{
  "baseAnalyzerId": "prebuilt-document",
  "models": { "completion": "gpt-5.2", "embedding": "text-embedding-3-large" },
  "description": "Supplier invoices from European vendors",
  "config": { "returnDetails": true, "estimateFieldSourceAndConfidence": true },
  "fieldSchema": {
    "fields": {
      "VendorName":   { "type": "string", "method": "extract",  "description": "Legal name of the vendor" },
      "InvoiceDate":  { "type": "date",   "method": "extract",  "description": "Invoice issue date" },
      "TotalAmount":  { "type": "number", "method": "extract",  "description": "Total including tax, in invoice currency" },
      "PaymentTerms": { "type": "string", "method": "generate", "description": "Payment terms summarized in one sentence" }
    }
  }
}

This is a creation-request body for a GA document analyzer: supply supplierInvoice in the analyzer URL, not as an analyzerId property in the body. Configure resource model-deployment mappings first and choose models listed as supported for that analyzer. Submit analysis separately, poll to a terminal result and handle missing fields, warnings, throttling and failures.

Key ideas:

• extract pulls a value that is present in the document; generate lets the model produce a derived or summarized value; classify selects from a fixed list. Choosing extract for literal values reduces invention.
• Good field descriptions act as prompts for each field.
• Enable document field source/confidence estimation for review. The extract method requires source/confidence estimation; it is not supported by the current agentic preview. Images, audio and video do not generally expose the same field-confidence controls.
• Content Understanding uses model deployments in your Foundry resource (a chat model and an embedding model), so quotas and costs of those deployments apply as well as analysis charges.

4.4 Calling an analyzer

The usual pattern is asynchronous: submit the file (by URL or binary upload), receive an operation location, poll until it succeeds, then read the result containing Markdown and fields. Because analysis can take seconds to minutes, batch large workloads with queues or storage-triggered functions and handle retries and throttling.

4.5 Standard versus pro (and agentic) processing

The syllabus still mentions pro mode, but it belonged to the retired 2025-05-01-preview API. Standard extraction remains available on the 2025-11-01 GA API. In the 2026-06-01-preview document API, config.workflow = "agentic" enables additional reasoning over one input file, potentially containing related documents. It does not support extract-method fields or labeled-sample improvement and is not a drop-in replacement for old multi-file/reference-data pro configurations.

For an invoice, purchase order and contract stored separately, extract each with GA analyzers and reconcile validated fields in application code. Consider the document agentic preview only within its one-file scope, with an evaluated fallback and approval before consequential writes.

4.6 Layout analysis, Document Intelligence and Content Understanding

Both services evolve, and Content Understanding is positioned as the generative, multimodal option. On the exam, match by requirement: structured fields across many content types with a schema → Content Understanding; OCR only → OCR/Read.

Worked example 3: invoices into an ERP

Scenario. An accounts-payable team receives 15,000 invoices a month as PDFs, photos from phones and Excel exports, in several languages. They need vendor, date, line items and total in the ERP. Wrong totals cost money; low-confidence invoices must be reviewed.

Reasoning.

1. Use a Content Understanding analyzer (the prebuilt invoice analyzer, extended by a custom analyzer if extra fields are required) so one schema serves PDFs and images.
2. Enable confidence and source grounding.
3. Validate in code: reconcile subtotal, taxes, discounts, shipping and rounding against the total; check currency, date and the vendor master.
4. Route invoices below a confidence threshold, or failing validation, to a reviewer UI that highlights the source region.
5. Store Markdown and fields; write approved results to the ERP using an idempotency key to avoid duplicates.
6. Measure field-level accuracy on 300 labelled invoices per language and monitor drift.

Why not a bare chat call for each invoice? Supported models offer structured outputs, but document parsing, source locations, review handling and validation still require additional engineering. Why not OCR alone? OCR returns text, not fields and relationships between them.

For capture and privacy details, see multimodal understanding and visual safety and text analysis, translation and speech.

5. Multimodal ingestion: images, audio and video

Design tip: keep provenance in every chunk (source file, page or timestamp) so the application can show evidence and open the exact moment or page.

6. Connecting retrieval to agents

An agent decides when to retrieve. You expose retrieval as a tool:

• Built-in Azure AI Search tool for Foundry agents: attach an index (via a project connection) so the agent can query it, with query type and filter options.
• Knowledge sources / knowledge bases (agentic retrieval) in Azure AI Search: a GA minimal extractive path is available on API 2026-04-01. LLM query planning, answer synthesis and conversational reasoning remain preview capabilities on the corresponding preview API. Choose the API and reasoning configuration explicitly rather than assuming every knowledge base plans queries.
• Function tools that wrap your own search or SQL calls when you need custom logic.
• MCP servers that expose knowledge systems to agents through a standard protocol.

Design guidance:

1. Describe the tool precisely ("Search the HR policy handbook; use for leave, benefits and conduct questions") so the model picks it correctly.
2. Enforce access in a trusted backend: resolve the caller’s authorized groups and construct the filter there. Managed identity authenticates a service, not automatically the end user. If a managed search tool cannot enforce per-user permissions on every query, use an authorized function or an appropriately configured native ACL path.
3. Require citations in answers and return source references from the tool.
4. Limit result size to control tokens; return the best few chunks with metadata rather than dumping the index.
5. Evaluate groundedness, retrieval relevance and citation correctness on a fixed set; trace tool calls to debug misses.
6. Treat retrieved text as untrusted: documents can contain injected instructions, so apply prompt-attack protection and least privilege.

Comparison: choosing a retrieval or extraction option

Common pitfalls

• Using a different embedding model for queries than for documents.
• Skipping OCR for scanned files, then wondering why the index is empty for them.
• Chunking by a fixed size through tables and clauses instead of by structure.
• Relying on vector search alone for identifiers and codes.
• Believing the semantic ranker finds documents that retrieval missed; it only reranks results.
• Putting access control in the prompt instead of in a filter.
• Treating generated field values as facts without validation, confidence or grounding.
• Forgetting deletion detection so removed documents remain searchable.
• Choosing a preview feature for a critical production path without a fallback.
• Not recording page numbers or timestamps, making citations impossible.

Exam tips

• Mixed exact-code and paraphrase requirements → evaluate hybrid search with semantic reranking.
• "Scanned or image-only PDFs" → OCR / layout skill in the skillset.
• "Query text should be embedded automatically" → integrated vectorization with a vectorizer.
• "Keep the documents' tables and headings for chunking" → layout analysis to Markdown, then structure-aware split.
• "Fields from invoices, forms and photos with confidence" → Content Understanding analyzers (or prebuilt invoice model) with grounding.
• "Search a video for a moment" → segment descriptions and transcripts with timestamps in the index.
• "Users must see only their documents" → security filter on group identifiers, not prompt instructions.
• "Agent needs company knowledge" → an authorized retrieval tool or knowledge base that enforces caller permissions before returning evidence.

Mini case study: Northwind Insurance Knowledge Platform

Northwind Insurance wants one assistant that answers staff questions using: 120,000 policy PDFs (many scanned), 5,000 photos of damaged property with adjuster notes, 3,000 recorded claim calls, and 200 training videos. Requirements: citations with page or timestamp, staff see only their region's documents, claim-form fields extracted into the claims system with a review step for uncertain values, and supported service-to-service calls use managed identity with least-privilege roles.

Task. Design the pipeline.

Solution.

1. Policy PDFs: indexer + skillset with the Document Layout/OCR skill, structure-aware split, integrated vectorization; fields page, policyId, region, groups, effectiveDate.
2. Photos: image analyzer producing descriptions and fields such as damage_type; index description text and keep a pointer to the original photo; the adjuster's note is indexed with it.
3. Calls: batch transcription with diarization or an audio analyzer, PII redaction, summary and fields (claim_number, complaint_reason); chunk by time window with timestamps.
4. Videos: video analyzer with scene segments, transcripts and descriptions indexed with start/end times.
5. Search: hybrid plus semantic ranking; trusted backend code verifies identity, resolves region/group entitlements and creates security filters before retrieving chunks. Apply the same policy to citations and source-file links.
6. Forms: a custom Content Understanding analyzer for claim forms with confidence and grounding; validations in code; below-threshold results go to a reviewer UI.
7. Agent: use a retrieval function that enforces the verified user’s entitlements, or a managed Search/knowledge-base path only after confirming supported ACL behavior. Return source references, require citations, and evaluate retrieval and citation accuracy; optional query planning is a separate preview choice.
8. Operations: indexer monitoring, replica sizing, cost tracking per pipeline, content-safety and prompt-attack protection on retrieved text.

Key terms

• Indexer: an Azure AI Search component that loads data from a source into an index.
• Skillset: a pipeline of enrichment skills applied during indexing.
• Chunking: splitting content into retrievable pieces.
• Embedding: a numeric vector representing meaning.
• HNSW: an approximate nearest-neighbour index structure for vectors.
• Hybrid search: keyword and vector queries merged, commonly with RRF.
• RRF: Reciprocal Rank Fusion, a method for merging ranked lists.
• Semantic ranker: a language-model reranker for top results.
• Integrated vectorization: built-in chunking and embedding at indexing and query time.
• Analyzer: a Content Understanding configuration defining extraction.
• Grounding: pointing to where in the source a value or answer came from.
• Agentic retrieval: knowledge-base retrieval; LLM query planning and conversational synthesis require preview configurations.
• Security trimming: filtering results to what the caller may see.

Summary checklist

• I can design an ingestion pipeline for documents, images, audio and video with metadata for citations.
• I can choose between keyword, vector, hybrid and semantic search, and justify hybrid plus semantic ranking.
• I can configure indexers, skillsets, OCR/layout, chunking and integrated vectorization.
• I can use Content Understanding analyzers for Markdown and structured fields with confidence and grounding.
• I can distinguish GA extraction, retired pro mode and the one-file document agentic preview.
• I can implement security trimming and keep the index fresh.
• I can expose retrieval to an agent as a tool and evaluate it.

Sources

• Microsoft Learn, AI-103 study guide: https://learn.microsoft.com/en-us/credentials/certifications/resources/study-guides/ai-103 (opens in a new tab)
• What is Azure Content Understanding in Foundry Tools?: https://learn.microsoft.com/en-us/azure/ai-services/content-understanding/overview (opens in a new tab)
• Content Understanding service limits and migration notes: https://learn.microsoft.com/en-us/azure/ai-services/content-understanding/service-limits (opens in a new tab)
• Create a custom analyzer (tutorial): https://learn.microsoft.com/en-us/azure/ai-services/content-understanding/tutorial/create-custom-analyzer (opens in a new tab)
• Azure AI Search documentation: https://learn.microsoft.com/en-us/azure/search/ (opens in a new tab)
• Agentic mode and pro migration: https://learn.microsoft.com/en-us/azure/ai-services/content-understanding/concepts/agentic-mode (opens in a new tab)
• Analyzer schema and confidence: https://learn.microsoft.com/en-us/azure/ai-services/content-understanding/concepts/analyzer-reference (opens in a new tab)
• Index projections: https://learn.microsoft.com/en-us/azure/search/search-how-to-define-index-projections (opens in a new tab)
• Security trimming: https://learn.microsoft.com/en-us/azure/search/search-security-trimming-for-azure-search (opens in a new tab)
• Document Layout skill: https://learn.microsoft.com/en-us/azure/search/cognitive-search-skill-document-intelligence-layout (opens in a new tab)
• Built-in Content Understanding skill: https://learn.microsoft.com/en-us/azure/search/cognitive-search-skill-content-understanding (opens in a new tab)
• Integrated vectorization: https://learn.microsoft.com/en-us/azure/search/vector-search-integrated-vectorization (opens in a new tab)
• Blob deletion tracking: https://learn.microsoft.com/en-us/azure/search/search-how-to-index-azure-blob-changed-deleted (opens in a new tab)
• Agentic retrieval API scope: https://learn.microsoft.com/en-us/azure/search/agentic-retrieval-overview (opens in a new tab)
• Hybrid search overview: https://learn.microsoft.com/en-us/azure/search/hybrid-search-overview (opens in a new tab)
• Skillsets in Azure AI Search: https://learn.microsoft.com/en-us/azure/search/cognitive-search-working-with-skillsets (opens in a new tab)