Joule
Search
⌃K
Comment on page

Time based metrics

The metrics engine can be leveraged to provide greater insights over a larger data set. These metrics can be used within custom processors to drive complex use case
This use case provides a simple demonstration on how metrics generated by the internal metrics engine can be emitted to connected publisher sinks.

Use case configuration

File: app-metrics-file.env
SOURCEFILE=${PWD}/conf/sources/stockQuoteStream.yaml
ENGINEFILE=${PWD}/conf/usecases/metricsAnalyticsExample.yaml
PUBLISHFILE=${PWD}/conf/publishers/fileStandardAnalytics.yaml

Pipeline configuration

The pipeline configures the metrics computation cycle of 1 minute, metrics to be generated, filtering and the select expression which emits the group by metrics to the connected publishing sink.
processing unit:
metrics engine:
policy:
timeUnit: MINUTES
frequency: 1
startup delay: 2
​
foreach metric compute:
metrics:
-
name: BidMovingAverage
metric key: symbol
table definition: standardQuoteAnalyticsStream.BidMovingAverage (symbol VARCHAR, avg_bid_min FLOAT, avg_bid_avg FLOAT,avg_bid_max FLOAT)
query:
SELECT symbol,
MIN(bid) AS 'avg_bid_min',
AVG(bid) AS 'avg_bid_avg',
MAX(bid) AS 'avg_bid_max'
FROM standardQuoteAnalyticsStream.quote
WHERE
ingestTime >= date_trunc('minutes',now() - INTERVAL 2 MINUTES) AND ingestTime <= date_trunc('minutes',now())
GROUP BY symbol
ORDER BY 1;
truncate on start: true
compaction:
frequency: 8
timeUnit: HOURS
​
pipeline:
- filter:
expression: "symbol != 'A'"
​
select:
expression: "symbol,
BidMovingAverage.avg_bid_max;WHERE symbol=${symbol} 'avg_bid_max'"