# Metrics engine

## Overview

Staying informed with near real-time business metrics allows organisations to make swift, decisive actions that enhance success. Joule’s SQL-compliant metrics engine facilitates this by generating real-time analytics and storing metrics through SQL expressions, supporting KPIs, alerting and predictive insights. 

By providing a framework for proactive management, Joule enables organisations to focus on clear, measurable goals and make data-driven decisions. The engine computes metrics using event data and runtime policies, empowering users to analyse and react to data effectively as it streams.

## Use cases

Here are some use cases of how metrics it can be applied in stream processing:

1. <mark style="color:green;">**Business metrics monitoring**</mark>\
   Track intra-day business metrics such as number of orders placed in an hour.
2. <mark style="color:green;">**Proactive service failure alerting**</mark>\
   Track key failure indicators to raise early failure alerting.
3. <mark style="color:green;">**Data quality metrics**</mark>\
   Monitor data quality in near real-time.
4. <mark style="color:green;">**ML feature engineering**</mark>\
   Use calculated metrics as feature vector components.
5. <mark style="color:green;">**KPI analytics**</mark>\
   Apply metrics to form intra-day KPIs. For example *Conversion rate* uses the number of conversions and total audience are running metrics to form the result.

## Key features

* ANSI SQL compliant metrics calculation definition
* Apply calculated metrics within processors
* Pre-computed metrics import
* Runtime policy&#x20;
* Metric management

{% hint style="info" %}
Powered by the [DuckDB](https://duckdb.org) In-memory column database
{% endhint %}

## Example

The following example computes a single family of metrics `BidMovingAverage` at set time intervals and saved to an in-memory `standardQuoteAnalyticsStream.BidMovingAverage` SQL table.

The `emit` definition performs a query lookup against the table and returns the `avg_bid_max` for each matching symbol.

```yaml
processing unit:
  metrics engine:
    runtime policy:
      frequency: 1
      startup delay: 2
      time unit: MINUTES

    foreach metric compute:
      metrics:
        - name: BidMovingAverage
          metric key: symbol
          table definition: standardQuoteAnalyticsStream.BidMovingAverage 
          (symbol VARCHAR, avg_bid_min FLOAT, avg_bid_avg FLOAT,avg_bid_max FLOAT)
          query:
            SELECT symbol,
            MIN(bid) AS 'avg_bid_min',
            AVG(bid) AS 'avg_bid_avg',
            MAX(bid) AS 'avg_bid_max'
            FROM standardQuoteAnalyticsStream.quote
            WHERE
            ingestTime >= date_trunc('minutes',now() - INTERVAL 2 MINUTES) AND 
            ingestTime <= date_trunc('minutes',now())
            GROUP BY symbol
            ORDER BY 1;
          truncate on start: true
          compaction policy:
            frequency: 8
            time unit: HOURS

emit:
  select: "symbol, BidMovingAverage.avg_bid_max;WHERE symbol=${symbol} 'avg_bid_max'"

group by:
  - symbol
```

## Available metric engine options

In the following articles you can further explore how to interact with the metrics engine.

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th></th><th data-hidden></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><mark style="color:orange;"><strong>Create a metric</strong></mark></td><td>Define time based metrics using captured events</td><td></td><td><a href="metrics-engine/create-metrics">create-metrics</a></td></tr><tr><td><mark style="color:orange;"><strong>Apply metrics</strong></mark></td><td>Apply metrics within stream processing functions</td><td></td><td><a href="metrics-engine/apply-metrics">apply-metrics</a></td></tr><tr><td><mark style="color:orange;"><strong>Manage metrics</strong></mark></td><td>Apply runtime compaction policies on calculated metrics </td><td></td><td><a href="metrics-engine/manage-metrics">manage-metrics</a></td></tr><tr><td><mark style="color:orange;"><strong>Priming metrics</strong></mark></td><td>Prime metrics with existing values on process startup</td><td></td><td><a href="metrics-engine/priming-metrics">priming-metrics</a></td></tr></tbody></table>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.fractalworks.io/joule/components/analytics/metrics-engine.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.