A document answer can sound grounded while retrieving the wrong version, the wrong section, an obsolete policy, a neighboring tenant's content, or a passage that does not support the final claim. Adding a citation field can make the failure look more credible. Production RAG quality depends on content boundaries, index design, retrieval evidence, identity, and answer evaluation working together.
Microsoft's current source-specific documentation describes a preview connection from Fabric Data Agent to an Azure AI Search index built in Microsoft Foundry. The Data Agent can retrieve unstructured content and combine it with structured sources. Because some broader concept documentation still carries older unstructured-data limitations, confirm tenant availability and behavior in the target region and capacity before promising a rollout.
Which document-backed decision needs a faster, evidence-linked answer? Start with that domain, not every file the organization owns.
Define the document answer contract
Select one document domain, user cohort, accountable content owner, decisions supported, authoritative repositories, prohibited content, freshness target, citation standard, access boundary, and measurable outcome. Define when the answer must quote or cite evidence, ask a clarifying question, return no answer, or escalate to a person.
| Contract area | Required decision | Evidence | Failure to prevent |
|---|---|---|---|
| Content scope | Document types, repositories, owners, versions, languages, classifications, retention, and exclusions. | Approved corpus inventory and authority rules. | Draft, obsolete, duplicate, or unauthorized content becomes evidence. |
| Question scope | Supported intents, required qualifiers, prohibited advice, no-answer behavior, and escalation. | Representative question and refusal catalogue. | Agent answers beyond the indexed evidence or intended decision. |
| Retrieval | Search type, chunking, fields, enrichment, filters, document count, ranking, and citation locations. | Retrieval ground truth with relevant chunks. | Relevant document exists but the wrong passage reaches the answer. |
| Identity | User and service identities, Azure AI Search roles, document access, sharing channel, and denied cases. | Effective-access matrix and negative tests. | Data Agent access is mistaken for permission to every indexed document. |
| Quality | Retrieval recall, ranking, groundedness, citation correctness, completeness, latency, and severity thresholds. | Versioned evaluation suite and release gate. | Fluent, cited answer passes despite unsupported material claims. |
| Preview gate | Target tenant support, environment, fallback, SLA assumption, expansion criteria, and risk owner. | Go, limit, or stop recommendation. | Preview and documentation drift become a production commitment. |
Design an index the agent can retrieve safely
Clean the corpus before tuning retrieval. Resolve duplicate and superseded documents, define version precedence, preserve effective dates and owners, exclude unsupported or sensitive material, and keep stable source identifiers. OCR, layout extraction, tables, images, footnotes, and scanned attachments need explicit preprocessing and acceptance tests; an index cannot retrieve content it parsed incorrectly.
Choose chunk boundaries that preserve the unit of meaning needed by the question. Keep headings, document identity, section path, version, effective date, classification, tenant or audience, and source location as retrievable metadata. Test whether adjacent chunks or parent context are required for definitions, exceptions, procedures, and tables. More chunks are not automatically better context.
Microsoft exposes full-text, hybrid, and semantic search choices and recommends retrieving 3–20 documents per query. Treat those as tunable variables, not defaults to accept blindly. Compare search types and document counts on the same question set, measuring relevant-chunk recall, irrelevant context, latency, answer quality, and processing cost.
Design citation fields before indexing. Microsoft says citations appear only when at least one supported case-sensitive field exists: url, sourceUrl, filePath, path, or folderPath. Use a durable location a reviewer can open and tie it to document version and section metadata. A link to the correct file is still an incorrect citation if the cited passage does not support the claim.
Evaluate retrieval, grounding, and source routing
Create ground truth with real user questions and an approved set of relevant documents and passages. Preserve expected answer points, unacceptable claims, citation targets, access persona, source route, freshness, and severity. Include synonyms, acronyms, version conflicts, tables, exceptions, no-answer questions, adversarial instructions inside documents, inaccessible content, and questions that need both a structured metric and an unstructured explanation.
Use the Data Agent source context to describe the index, key fields, authority, and intended question types. Use agent instructions to define routing, evidence requirements, conflict handling, citation behavior, and no-answer policy. Build agent with AI does not currently support unstructured sources, so manually review these settings and validate every change against the same evaluation set.
| Dimension | Test | Pass condition | Remediation |
|---|---|---|---|
| Retrieval recall | Whether authoritative passages appear in the retrieved set across wording, acronyms, and filters. | Required evidence is retrieved for every critical supported question. | Corpus cleanup, enrichment, fields, chunking, synonyms, hybrid or semantic search. |
| Ranking precision | Relevant versus obsolete, duplicate, neighboring, or weakly related chunks. | Top context prioritizes current authoritative evidence. | Version metadata, filters, scoring, semantic configuration, or fewer documents. |
| Groundedness | Every material answer claim against retrieved text. | No claim exceeds or contradicts the available evidence. | Stricter instructions, better context, refusal, or human escalation. |
| Citation fidelity | Document, version, section, passage, and link for each material claim. | Citation opens the evidence that supports the exact claim. | Index location fields, metadata, answer format, or claim-level evaluation. |
| Routing | Unstructured index versus SQL, semantic, KQL, graph, or ontology source selection. | Each question uses the authoritative source or an explicit combination. | Source context, agent instructions, domain separation, or clarification. |
| Access | Allowed and denied users, roles, sensitive documents, channel identity, and permission changes. | Unauthorized content is never retrieved or exposed in answer or citation. | Azure AI Search RBAC and index security correction, source separation, or channel redesign. |
| Operations | Freshness, failed indexing, partial corpus, latency, throttling, dependency outage, and cost. | Service stays within SLOs and fails with a clear limitation. | Monitoring, ingestion recovery, capacity tuning, fallback, or stop rule. |
Prove identity and operate the retrieval chain
Enable role-based authentication on Azure AI Search and test the exact identity used by the consumption channel. Microsoft's setup guidance lists Search Index Data Contributor and Search Index Data Reader for the connecting user or service principal and says the asking user's identity is sent to the index. Validate least privilege, denied users, permission removal, group changes, mixed-source questions, and whether citations reveal metadata a user should not see.
Monitor source ingestion, failed and skipped documents, parsing quality, index freshness, index schema version, retrieval latency, search type, document count, retrieved chunks, citation coverage, answer corrections, denied requests, Data Agent routing, capacity, and Azure AI Search consumption. Keep sensitive prompt and document content out of broad logs by default while retaining enough correlation metadata to reproduce failures.
Test search unavailable, index empty, stale index, malformed document, missing citation field, permission revoked, retrieval timeout, throttling, irrelevant context, conflicting documents, and structured source unavailable. The answer should expose the limitation or abstain rather than fabricate a synthesis. Maintain a known-good index and Data Agent configuration with rollback and re-evaluation steps.
Run a three-to-five-week unstructured RAG assessment
- Select one document-backed decision, corpus, user cohort, content owner, risk boundary, and measurable outcome.
- Inventory authority, versions, duplicates, formats, access, classifications, freshness, OCR and enrichment needs, and existing search behavior.
- Build or refine the Azure AI Search index, metadata, chunking, citation fields, RBAC, ingestion, and observability.
- Connect the index by resource URL; configure context, search type, document count, routing, evidence, conflict, and no-answer instructions.
- Create retrieval and answer ground truth; run relevance, citation, access, mixed-source, stale-data, adversarial, outage, latency, and cost tests.
- Classify failures by corpus, parsing, enrichment, index schema, retrieval, routing, permission, answer, runtime, or documentation drift; remediate and repeat.
- Deliver the corpus contract, index design, evaluation scorecard, access matrix, runbooks, preview decision, and go, limit, or stop recommendation.
Frequently asked questions
Can Microsoft Fabric Data Agent use Azure AI Search for unstructured data?
Yes, in preview. Microsoft's current source-specific documentation describes connecting an Azure AI Search index built in Microsoft Foundry to a Fabric Data Agent by resource URL so the agent can retrieve content from PDFs, text files, and other indexed documents and combine it with structured sources.
How does Fabric Data Agent preserve Azure AI Search permissions?
Microsoft documents role-based authentication and says the asking user's identity is sent to the Azure AI Search index so the resource's access controls and permissions are respected. Test the exact user, service principal, index roles, consumption channel, and denied cases rather than assuming access from either product alone.
How do citations work with Fabric Data Agent and Azure AI Search?
Citations can appear when the index contains at least one supported case-sensitive location field: url, sourceUrl, filePath, path, or folderPath. Citation presence is not enough; evaluation should verify that each citation identifies the document and passage that actually supports the material claim.
Does Build agent with AI configure Azure AI Search sources?
No. Microsoft's current Build agent with AI preview is limited to SQL and Eventhouse data sources and lists unstructured data as unsupported. Azure AI Search context, search type, document count, routing instructions, and response behavior need manual design and evaluation.
How long does a Fabric Data Agent Azure AI Search assessment take?
A focused assessment commonly takes three to five weeks for one document domain, one search index, one Data Agent, and a representative question set. It covers content inventory, index and citation design, identity, retrieval quality, answer groundedness, mixed-source routing, failure tests, preview risk, and operating controls.
Official implementation references
- Connect Fabric Data Agent to an Azure AI Search index
- Configure unstructured sources in Fabric Data Agent
- Build agent with AI supported source limitations
- Azure AI Search retrieval architecture
- Azure AI Search role-based access control
- Azure AI Search RAG patterns and retrieval choices
Start with one document domain, its real questions, and the passages that should answer them. Datrick can turn that evidence into a governed index, Data Agent source, retrieval evaluation, and rollout recommendation.