Real-time data makes a plausible wrong answer especially dangerous. A question about failures in the last hour can accidentally use ingestion time instead of event time, scan thirty days, merge tenants, count retries as incidents, ignore late arrivals, or summarize a result that has already exceeded the conversational output limit. The generated KQL may be syntactically valid and still misrepresent the operational state.
Fabric Data Agent connects to an Eventhouse KQL database and uses NL2KQL to generate, validate, and execute read-only queries over selected schema entities. Microsoft documents support for tables, materialized views, functions, and shortcuts, plus source instructions and example queries. A production design must turn those capabilities into a deliberately bounded operational data product.
Which operational decision should become faster, and what exact event-time window proves it? Start there, not with the entire Eventhouse schema.
Define the real-time question contract
Select one event domain, operational decision, user cohort, Eventhouse source, supported questions, prohibited uses, freshness target, default time window, identity, and accountable owner. Define whether the answer is for investigation, situational awareness, trend analysis, or a production response decision. A Data Agent is read-only; a generated answer should not directly trigger an operational action.
| Contract area | Required decision | Evidence | Failure to prevent |
|---|---|---|---|
| Event domain | Systems, event types, entities, environments, tenants, owners, and authoritative Eventhouse database. | Domain map and approved question catalogue. | One generic agent mixes telemetry with different meanings and owners. |
| Time semantics | Event time, ingestion time, timezone, default and maximum windows, late arrival, retention, and freshness. | Deterministic KQL for each critical time-window pattern. | Correct query shape over the wrong timestamp or operational period. |
| Entity semantics | Incident, request, session, device, retry, state transition, deduplication key, and current-state rule. | Grain and lifecycle definitions approved by operations owners. | Counting rows or retries when the question asks for distinct incidents. |
| Selected schema | Tables, materialized views, functions, and shortcuts required for the question set. | Minimal schema inventory with purpose, owner, and permission. | Broad schema increases ambiguity, query cost, and sensitive-data exposure. |
| Quality | Expected KQL, result, clarification, latency, consumption, and severity-weighted threshold. | Versioned ground-truth and regression suite. | Fluent summary accepted without reviewing generated KQL and result. |
| Operations | Identity, capacity, monitoring, incident, escalation, release, rollback, and support. | Runbooks and production-shaped failure tests. | Unbounded or unauthorized query becomes an Eventhouse incident. |
Design a KQL source for conversational investigation
Select only the Eventhouse entities needed for the supported questions. Prefer materialized views for recurring aggregations or normalized operational states, and approved KQL functions for complex logic that should not be reinvented by generated queries. Functions can encode deduplication, sessionization, health classification, or business calendars while retaining a reviewable query contract. Keep function parameters explicit and test their permissions and performance.
Use business-readable names and descriptions. Document timestamp columns, entity IDs, status and severity values, environment, region, tenant, units, dynamic fields, ingestion transformations, update policy behavior, and retention. Explain whether null means unknown, not applicable, or not yet processed. If the system uses abbreviated values, supply those formats in source instructions and examples because the agent cannot infer every row value before generating KQL.
Make time filtering unavoidable. Include a bounded default only when the business has approved it, otherwise require a clarifying question. Specify which timestamp applies to each question, how to interpret relative periods, and how to treat late or duplicated events. Encourage explicit time windows for high-volume sources and verify that every generated KQL query applies the correct predicate early enough to control scan and latency.
Build a ground-truth NL2KQL evaluation
Write source instructions for purpose, supported questions, table and view roles, join or lookup logic, event grain, state transitions, time rules, values, required filters, functions, and exclusions. Use question-KQL examples for representative patterns such as distinct incidents, failure rate, percentile latency, latest state, anomaly windows, top contributors, sequence analysis, and deployment correlation. Every example must validate against the selected schema and return the intended result.
Use real operational questions with fixed data snapshots or stable historical windows. For each, preserve the expected interpretation, approved KQL, result, grain, time range, identity, acceptable answer, and severity. Include ambiguity, unsupported requests, cross-tenant attempts, no-data periods, schema changes, late arrivals, high cardinality, and deliberately expensive questions.
| Dimension | Test | Pass condition | Remediation |
|---|---|---|---|
| Time correctness | Event versus ingestion time, timezone, relative window, late arrival, bin size, and ordering. | KQL uses the approved timestamp and exact operational period. | Source instruction, function, materialized view, or clarifying question. |
| Entity and state | Distinct IDs, retries, latest state, transitions, deduplication, and active records. | Result grain matches the decision and avoids double counting. | Governed function/view or representative few-shot. |
| KQL logic | Filters, summarize, joins, lookups, arg_max, percentiles, series, dynamic fields, and null handling. | Generated KQL reproduces approved deterministic results. | Reduce schema, clarify values, improve examples, or expose an approved function. |
| Security | Reader roles, restricted tables, sensitive fields, tenants, workspaces, and prompt manipulation. | Every persona sees only authorized data and denied requests fail closed. | Permission correction, separate source/agent, Purview control, or channel redesign. |
| Performance | Time predicate placement, scanned extent, materialized view use, result cardinality, latency, and concurrency. | Correct answer stays within query, capacity, and user-experience thresholds. | Bound windows, optimize KQL, materialize, route extracts elsewhere, or reject the question. |
| Answer fidelity | Units, window, data freshness, truncation, uncertainty, and attribution. | Natural-language answer preserves the KQL result and critical qualifiers. | Response instruction, explicit evidence, result-size warning, or human escalation. |
Do not use conversational output for bulk export. Microsoft documents result limits and summarization behavior for Data Agent responses. A question that needs all events, forensic preservation, or a complete dataset belongs in a governed query, notebook, report, export, or incident workflow. The Data Agent should help investigators find and understand bounded evidence.
Prove identity, capacity, and incident behavior
Test the actual consumption channel and user identity. Eventhouse users need the appropriate KQL database access, and source policies continue to apply. Microsoft currently documents that service-principal authentication is not supported for Data Agents connected to KQL databases. Do not build unattended custom-app access around that unsupported path; select a supported user identity or redesign the boundary.
Keep the Data Agent workspace and source capacity region compatible. Microsoft notes that a Data Agent cannot query a source when their workspace capacities are in different regions. Test source unavailable, permission revoked, schema changed, function changed, capacity pressure, timeout, query rejected, and result truncated. The user should receive a clear limitation rather than a fabricated operational conclusion.
Monitor question volume, generated-query failures, scanned data where available, latency, capacity, authorization errors, wrong time windows, high-cardinality requests, corrections, and negative feedback. Record the agent version, source configuration, identity, generated KQL, result metadata, and correlation ID without logging sensitive raw content by default. Maintain a known-good configuration and release path for instructions, examples, functions, and selected schema.
Run a three-to-five-week Eventhouse assessment
- Select one real-time decision domain, Eventhouse database, user cohort, operational owner, supported questions, risk, and measurable outcome.
- Map event grain, timestamps, entities, states, values, retention, refresh, source permissions, regions, capacity, and current KQL patterns.
- Choose the minimum tables, materialized views, functions, and shortcuts; document their purpose and optimize recurring logic.
- Write KQL-specific source instructions and validated question-KQL examples for time, state, rate, percentile, sequence, and correlation patterns.
- Build deterministic ground truth and run NL2KQL, identity, adversarial, no-data, high-volume, timeout, capacity, and truncation tests.
- Classify failures by source, schema, instruction, example, generated KQL, permission, runtime, or answer; remediate and repeat regression.
- Deliver the source contract, KQL semantic map, functions and examples, evaluation scorecard, security matrix, runbooks, and go, limit, or stop recommendation.
Frequently asked questions
Can a Microsoft Fabric Data Agent query an Eventhouse KQL database?
Yes. A Fabric Data Agent can attach an Eventhouse KQL database and translate natural-language questions into read-only KQL. The selected schema can include approved tables, materialized views, functions, and shortcuts. The generated query is validated against that scope and executed in place against Eventhouse.
How do we improve Fabric Data Agent NL2KQL accuracy?
Limit the selected schema, define event grain and time semantics, document entity keys and categorical values, provide source-specific instructions, use validated question-KQL examples, expose approved KQL functions for complex logic, and compare generated KQL and results with deterministic ground truth across representative questions.
Why are time filters critical for a Fabric Data Agent over Eventhouse?
Eventhouse often contains high-volume time-series data. An omitted or incorrect time filter can return the wrong operational period, scan unnecessary data, increase latency and consumption, or produce a summarized response that hides the problem. Define default windows, event-time columns, timezone, late-arrival handling, and clarification rules.
Does Fabric Data Agent service principal authentication support KQL sources?
Microsoft's current service-principal preview documentation says service principal authentication is not yet supported for Data Agents connected to a KQL database. Use and test a supported user identity path for the intended channel, and do not design unattended production access around an unsupported assumption.
How long does a Fabric Data Agent Eventhouse assessment take?
A focused assessment commonly takes three to five weeks for one Eventhouse domain, one Data Agent, and a representative operational question set. The work covers KQL source design, schema and function scope, identity, examples, ground truth, query cost, answer evaluation, failure tests, release controls, and operating runbooks.
Official implementation references
- Configure Eventhouse KQL sources in Fabric Data Agent
- Fabric Data Agent NL2KQL, governance, and limitations
- Question-query examples and evaluation
- Data Agent source configuration best practices
- Data Agent service-principal limitations for KQL sources
- AI agent options for Fabric Real-Time Intelligence
Start with one incident or service-health question set and its approved KQL. Datrick can turn that operational truth into a focused Data Agent source, evaluation suite, and production recommendation.