Streaming Data and Real-Time Intelligence

An Eventhouse is a specialized container for KQL databases that handle large volumes of continuous, append-oriented and time-series data. Data is commonly indexed by ingestion time and is usually queried rather than updated in place.

Eventhouse capabilities

• Ingest from files, cloud storage, OneLake, Kafka, Azure Event Hubs, and other streaming sources.
• Use database shortcuts to query supported external data without copying it.
• Make data available to other Fabric experiences through OneLake integration.
• Query with Kusto Query Language (KQL) or supported T-SQL experiences.

KQL pipeline model

KQL uses the pipe character (|) to pass data through operators such as where, project, summarize, join, and limit. The KQL queryset is used to author, save, visualize, and share queries.

KQL performance depends heavily on reducing the amount of data scanned and processed.

• Filter early, especially with a time-based where clause, so indexes can eliminate irrelevant records.
• Use project to return only the columns needed by the next step.
• Put the smaller input on the left side of a join when that improves the join's memory use.
• Use limit while exploring large result sets, especially after an aggregation.
• Use materialized views for recurring aggregations over large, continuously changing tables.
• Use stored functions to encapsulate reusable and parameterized KQL logic.

Eventstreams capture events from sources, optionally transform them, and route them to destinations.

Eventstream flow

• Sources can include Azure Event Hubs, IoT Hub, Kafka, MQTT, Service Bus, and Fabric events.
• Transformations can filter, manage fields, aggregate, group by a time window, union, join, or expand data.
• Destinations can include Eventhouse, Lakehouse, Activator, a derived stream, or a custom endpoint.

Real-Time Dashboards

• Tiles execute KQL queries and refresh automatically
• Base queries allow several tiles to reuse the same core query logic
• Parameters make dashboards interactive for viewers
• Multiple pages organise complex operational monitoring scenarios

Activator

Activator detects conditions in changing data and performs actions, following a Connect, Monitor, Act model.

• Object: a real-world entity such as a device, package or customer
• Property: a tracked attribute such as temperature or location
• Event: an incoming record that updates the object state
• Rule: the condition, threshold or pattern that triggers action
• Action: email, Teams, Power Automate, notebook or pipeline execution

Rule Design

• Use summarisation windows and duration conditions to avoid triggering on short-lived noise
• Property filters can narrow a rule to selected objects, priorities or locations