> For the complete documentation index, see [llms.txt](https://docs.fractalworks.io/joule/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.fractalworks.io/joule/components/analytics/metrics-engine.md).

# Metrics engine

## Overview

Staying informed with near real-time business metrics allows organisations to make swift, decisive actions that enhance success. Joule’s SQL-compliant metrics engine facilitates this by generating real-time analytics and storing metrics through SQL expressions, supporting KPIs, alerting and predictive insights. 

By providing a framework for proactive management, Joule enables organisations to focus on clear, measurable goals and make data-driven decisions. The engine computes metrics using event data and runtime policies, empowering users to analyse and react to data effectively as it streams.

## Use cases

Here are some use cases of how metrics it can be applied in stream processing:

1. <mark style="color:green;">**Business metrics monitoring**</mark>\
   Track intra-day business metrics such as number of orders placed in an hour.
2. <mark style="color:green;">**Proactive service failure alerting**</mark>\
   Track key failure indicators to raise early failure alerting.
3. <mark style="color:green;">**Data quality metrics**</mark>\
   Monitor data quality in near real-time.
4. <mark style="color:green;">**ML feature engineering**</mark>\
   Use calculated metrics as feature vector components.
5. <mark style="color:green;">**KPI analytics**</mark>\
   Apply metrics to form intra-day KPIs. For example *Conversion rate* uses the number of conversions and total audience are running metrics to form the result.

## Key features

* ANSI SQL compliant metrics calculation definition
* Apply calculated metrics within processors
* Pre-computed metrics import
* Runtime policy&#x20;
* Metric management

{% hint style="info" %}
Powered by the [DuckDB](https://duckdb.org) In-memory column database
{% endhint %}

## Example

The following example computes a single family of metrics `BidMovingAverage` at set time intervals and saved to an in-memory `standardQuoteAnalyticsStream.BidMovingAverage` SQL table.

The `emit` definition performs a query lookup against the table and returns the `avg_bid_max` for each matching symbol.

```yaml
processing unit:
  metrics engine:
    runtime policy:
      frequency: 1
      startup delay: 2
      time unit: MINUTES

    foreach metric compute:
      metrics:
        - name: BidMovingAverage
          metric key: symbol
          table definition: standardQuoteAnalyticsStream.BidMovingAverage 
          (symbol VARCHAR, avg_bid_min FLOAT, avg_bid_avg FLOAT,avg_bid_max FLOAT)
          query:
            SELECT symbol,
            MIN(bid) AS 'avg_bid_min',
            AVG(bid) AS 'avg_bid_avg',
            MAX(bid) AS 'avg_bid_max'
            FROM standardQuoteAnalyticsStream.quote
            WHERE
            ingestTime >= date_trunc('minutes',now() - INTERVAL 2 MINUTES) AND 
            ingestTime <= date_trunc('minutes',now())
            GROUP BY symbol
            ORDER BY 1;
          truncate on start: true
          compaction policy:
            frequency: 8
            time unit: HOURS

emit:
  select: "symbol, BidMovingAverage.avg_bid_max;WHERE symbol=${symbol} 'avg_bid_max'"

group by:
  - symbol
```

## Available metric engine options

In the following articles you can further explore how to interact with the metrics engine.

<table data-card-size="large" data-view="cards"><thead><tr><th></th><th></th><th data-hidden></th><th data-hidden data-card-target data-type="content-ref"></th></tr></thead><tbody><tr><td><mark style="color:orange;"><strong>Create a metric</strong></mark></td><td>Define time based metrics using captured events</td><td></td><td><a href="/pages/0ZM9YLCBCyxAepVnA5dt">/pages/0ZM9YLCBCyxAepVnA5dt</a></td></tr><tr><td><mark style="color:orange;"><strong>Apply metrics</strong></mark></td><td>Apply metrics within stream processing functions</td><td></td><td><a href="/pages/OMHJXGCZ5e5IGpYH6Usw">/pages/OMHJXGCZ5e5IGpYH6Usw</a></td></tr><tr><td><mark style="color:orange;"><strong>Manage metrics</strong></mark></td><td>Apply runtime compaction policies on calculated metrics </td><td></td><td><a href="/pages/vHMsc3LS0qAkvOmNjfCv">/pages/vHMsc3LS0qAkvOmNjfCv</a></td></tr><tr><td><mark style="color:orange;"><strong>Priming metrics</strong></mark></td><td>Prime metrics with existing values on process startup</td><td></td><td><a href="/pages/Q8NnzPQybKEGrQ0LO7I8">/pages/Q8NnzPQybKEGrQ0LO7I8</a></td></tr></tbody></table>