Streaming Analytics

Continuous analytic processing over streaming events

Joule offers a user-friendly streaming analytics capability tailored to deliver real-time insights efficiently. It enables users to harness existing analytical resources, define expressions, integrate reference data, and develop custom analytic plugins for complex calculations.

Key features

• Expression evaluation and execution

• Scripting and function execution

• Stateful calculations

• Calculation seeding with start values

• Constants and reference data

• Analytic result memory

Additionally developers can extend the analytical capabilities by implementing various analytics APIs. drive business the ability to execute imperative expressions, functions and scripts. Platform users can deploy custom stateful and stateless analytics using the Joule DSL using analytics processors.

Analytic Processor Overview

Available features

User Defined Analytic

Declarative stateful math expressions evaluated in real-time

User Defined Scripts

Leverage pre-existing analytics scripts within a streaming context

User Defined Functions

Extend the analytics ecosystem using the Analytics API

Javascript and Python are supported using the Graalvm runtime compilation engine

---

Worked example

This example demonstrates how two independent computed analytic pipeline results can be combined to generate an alerting signal.

Example analytics pipeline

Stage 1 - Pre-processing

Enrich event with signal thresholds using the internal in-memory database.

- enricher:
    fields:
      company_signals:
        by query: "select * from reference_data.nasdaq_companies where Symbol = ?"
        query fields: [ symbol ]
        with values: [ bid_signal_threshold, ask_signal_threshold ]
        using: JouleDB

Stage 2 - Event window analytics

Perform complex analytics using event based sliding windows and a stateful analytic function (exponential mean average).

- complex analytic:
  ema:
    parameters:
      ema_factor: 0.33333
  fields: [ bid,ask ]
  event history: 12

- complex analytic:
    ema:
      parameters:
        ema_factor: 0.33333
    fields: [ bid,ask ]
    event history: 26

Stage 3 - Analytics

Combine the results of the stateful analytics in to two basic difference calculations using Javascript notation/

- analytic:
    expression: ema12_ask - ema26_ask
    assign to: macd_ask_signal

- analytic:
    expression: ema12_bid - ema26_bid
    assign to: macd_bid_signal

Stage 4 - Alerting Rules

Finally once the calculations have been completed perform the final predicate using the having expression to send an alerting event to a downstream system

emit:
  select: "symbol, macd_bid_signal, macd_ask_signal"
  having: "macd_bid_signal <= bid_signal_threshold or macd_ask_signal >= ask_signal_threshold"