An Operations Agent can produce a convincing recommendation from a rule that monitors the wrong field, uses ingestion time instead of business event time, repeats every five minutes while a state remains true, or never observes a late or unsupported record. The message is downstream of detection. If detection is wrong, explanation and action quality cannot repair the control.
Microsoft exposes the query behind each generated rule and distinguishes state conditions from transition conditions. Treat that query and condition as executable production logic: version it, replay it against known history, verify source and time semantics, measure misses and false positives, and approve it independently from the natural-language instruction.
Can you show the exact query and source records that should have triggered the rule? If not, establish ground truth before changing the prompt.
Define the monitored signal and time contract
For each rule, record the business object, unique identifier, source table, property and underlying column, data type, unit, event timestamp, ingestion timestamp, expected arrival delay, population, grouping, window, threshold, exclusions, recovery, owner, severity, and response objective. Use flat Eventhouse tables with descriptive columns and identify the business object explicitly.
Microsoft says Operations Agent defaults to table ingestion time for identifying recent records and runs active rule queries every five minutes. A five-minute query cadence is not an immediate event trigger. Model event-to-ingestion delay, query-window overlap, source freshness, query duration, reasoning, Teams delivery, and capacity effects in the end-to-end detection objective.
Choose state or transition behavior deliberately
| Control | Required evidence | Failure risk | Release decision |
|---|---|---|---|
| Generated query | Full KQL or ontology query, selected property, underlying field, time filter, grouping, projection, and result shape. | The playbook looks correct while the query reads the wrong evidence. | Run the query independently and reconcile every control case. |
| State condition | Examples such as Is above, Is below, or Is; expected behavior while the value remains true. | The rule signals on repeated evaluations and creates duplicate alert burden. | Use only when recurring awareness of the ongoing state is intentional. |
| Transition condition | Examples such as Crosses above, Crosses below, Enters range, Exits range, or Becomes. | The first transition is missed because prior state, null, recovery, or object identity is wrong. | Use when one signal per state entry is required and test re-entry. |
| Threshold and unit | Numeric value, inclusive boundary, scale, unit, valid range, precision, and business owner approval. | Qualitative wording causes the generated rule to invent or misread a threshold. | Use explicit measurable conditions and boundary examples. |
| Object identity | Unique entity key, table, duplicates, nulls, reuse, aggregation level, and partition behavior. | State from one asset or process is applied to another or fragmented across records. | Prove stable identity and expected per-object state tracking. |
| Time window | Ingestion time, event time, lookback, late-arrival distribution, overlap, timezone, and reset behavior. | Records arrive outside the queried window or are reevaluated unexpectedly. | Align source and query time semantics and test delayed data. |
| Recovery and suppression | Return-to-normal criteria, re-entry, maintenance windows, known exceptions, cooldown, recipients, and escalation. | Persistent incidents spam Teams or suppression hides a genuine recurrence. | Define explicit recovery and measurable alert-burden limits. |
Rule order and separation also matter because the LLM generates the playbook from instructions. Put each condition on a separate line, list higher-priority rules first, quote field names with special characters, and replace qualitative phrases with approved numeric definitions. Then validate the generated executable query instead of assuming the instruction survived unchanged.
Replay known events, quiet periods, and timing edges
| Test | Method | Pass condition | Remediation |
|---|---|---|---|
| Known positive | Replay incidents with exact expected object, value, time, rule, and operation outcome. | Every required case triggers within the documented evaluation and delivery objective. | Source preparation, query, binding, window, threshold, condition, or permission fix. |
| Known negative | Replay normal periods, nearby values, excluded entities, maintenance windows, and expected exceptions. | No recommendation is created and alert burden stays below the owner-approved rate. | Threshold, exclusion, grouping, suppression, or separate rule. |
| Boundary | Test immediately below, equal to, and above every threshold plus null, zero, negative, and extreme values. | Inclusive and exclusive behavior matches the written contract. | Condition correction, unit normalization, or explicit clarification. |
| Persistent state | Keep the same object above or below the threshold across multiple five-minute evaluations. | Repeated signals occur only when the approved state policy requires them. | Transition condition, suppression, cooldown, or recipient aggregation. |
| Transition and recovery | Move false to true, remain true, return to normal, then re-enter; include null-to-true. | One signal per intended transition and a new signal only after valid recovery and re-entry. | Object key, prior-state logic, transition type, or recovery rule. |
| Late and duplicate data | Delay records, reorder events, duplicate identifiers, replay batches, and compare event with ingestion time. | No missed material case or duplicate operation outside the approved tolerance. | Ingestion pipeline, deduplication, query window, source model, or alternate control. |
| Missing data | Stop ingestion, omit identifiers, send null values, create sparse periods, and restore the stream. | The rule fails visibly or uses an approved absence condition; it doesn't imply normal operation. | Data-quality rule, heartbeat control, stop state, or escalation. |
| Query and access | Run generated KQL independently, inspect Query insights, remove creator source access, and test unsupported objects. | Query results reconcile and access failures are visible without false recommendations. | Query correction, regular table, permission, quoting, schema description, or rollout block. |
| Message load | Burst multiple objects and rules, sustain a true state, and observe Teams output under capacity pressure. | Owners receive actionable volume; simplified or throttled messages don't hide critical control state. | Rule prioritization, aggregation, suppression, capacity, alternate notification, or stop. |
Run in shadow mode beside the existing alert or operating process. Measure event-to-detection latency, recall, false-positive rate, duplicate operations per incident, missed recovery, Teams messages per owner, acknowledgment, recommendation utility, query cost, and capacity. Keep recommendation quality separate from rule accuracy.
Treat playbook regeneration as a monitored rule change
Version business goals, instructions, source definition, property mapping, generated query, condition type, threshold, window, grouping, suppression, recipients, actions, and agent state. A natural-language edit or playbook regeneration can alter executable monitoring behavior. Review the diff, rerun replay tests, record approval, and retain a rollback or stop path.
Use Query insights and the Operations Agent activity log to compare expected evaluations with actual query execution, recommendations, responses, and operation status. Alert when the agent is inactive, queries fail, data stops arriving, detection volume changes sharply, false positives rise, known control events are missed, Teams delivery is simplified or throttled, or capacity cost changes materially.
Maintain a rule registry with owner, purpose, source, time contract, query, state or transition semantics, threshold authority, regression dataset, release version, alert budget, dependencies, and retirement condition. Revalidate after schema, ingestion, KQL, ontology, instruction, runtime, recipient, region, capacity, or Fabric release changes.
Run a two-to-four-week rule accuracy assessment
- Select one Operations Agent, monitored process, source, rule set, accountable owners, current alert path, incident history, and required detection objective.
- Inventory source tables, fields, object IDs, event and ingestion time, data quality, generated queries, conditions, thresholds, grouping, suppression, recipients, agent state, and permissions.
- Build a labeled replay set with known positives, normal periods, boundary values, persistent states, transitions, recovery, late and duplicate records, missing data, and access failures.
- Execute generated queries independently, compare each result with ground truth, and classify failures by data, time, query, binding, condition, state, permission, capacity, or delivery.
- Refine source preparation, instructions, field descriptions, queries, thresholds, state or transition choice, suppression, recovery, recipient load, and monitoring.
- Run shadow-mode tests across multiple evaluation cycles and measure recall, false positives, duplicate operations, latency, message burden, recommendation utility, and cost.
- Deliver the signal contract, rule registry, verified queries, replay suite, scorecard, revised playbook, monitoring, release and rollback gates, and go, limited pilot, recommendation-only, or stop decision.
Frequently asked questions
Why is my Microsoft Fabric Operations Agent rule not triggering?
Common causes include a generated query or property binding that doesn't match the source, missing or unsuitable ingestion time, unsupported table or condition, an inactive agent, creator permission failure, no matching data in the evaluation window, or a state-transition assumption that doesn't match the configured condition. Inspect the rule query and run it against representative data before rewriting instructions.
Why does Fabric Operations Agent send duplicate Teams alerts?
A state condition can remain true across repeated evaluations and signal again while the value stays in that state. Microsoft documents a five-minute rule-query cadence. If the business needs one message only when the value enters the state, use an appropriate transition condition and test recovery and re-entry behavior.
What is the difference between state and transition conditions in Operations Agent?
A state condition is met whenever the current value satisfies the rule, such as Is above. A transition condition is met when the value changes from outside to inside the condition, such as Crosses above, Enters range, or Becomes. The correct choice depends on whether the team wants repeated awareness of an ongoing state or one signal when the state changes.
How often does Microsoft Fabric Operations Agent evaluate rules?
Microsoft's current documentation says an active Operations Agent runs each rule query every five minutes. Production expectations should include source ingestion delay, query duration, reasoning and Teams delivery time, capacity behavior, and the fact that a five-minute cadence isn't an immediate event trigger.
How long does an Operations Agent rule accuracy assessment take?
A focused assessment commonly takes two to four weeks for one agent, monitored source, rule set, and representative incident history. It covers source and time semantics, generated queries, state and transition logic, replay, false positives, missed events, duplicate alerts, monitoring, change control, and release gates.
Official implementation references
- Operations Agent generated queries, conditions, five-minute cadence, and activity log
- Operations Agent source, rule, runtime, and messaging limitations
- Operations Agent recommendations and Teams delivery
- Operations Agent responsible use and probabilistic behavior
- Operations Agent capacity and billing
Start with the rule that missed a known incident or keeps messaging the same unresolved state. Datrick can reconcile the query, replay the evidence, correct state and transition behavior, and establish measurable release gates.
When the failure is live and the fault domain is not yet known, contain and trace the Operations Agent incident across source, state, operation, Teams, identity, action, capacity, and recent change evidence.