Joule
  • Welcome to Joule's Docs
  • Why Joule?
    • Joule capabilities
  • What is Joule?
    • Key features
    • The tech stack
  • Use case enablement
    • Use case building framework
  • Concepts
    • Core concepts
    • Low code development
    • Unified execution engine
    • Batch and stream processing
    • Continuous metrics
    • Key Joule data types
      • StreamEvent object
      • Contextual data
      • GeoNode
  • Tutorials
    • Getting started
    • Build your first use case
    • Stream sliding window quote analytics
    • Advanced tutorials
      • Custom missing value processor
      • Stateless Bollinger band analytics
      • IoT device control
  • FAQ
  • Glossary
  • Components
    • Pipelines
      • Use case anatomy
      • Data priming
        • Types of import
      • Processing unit
      • Group by
      • Emit computed events
      • Telemetry auditing
    • Processors
      • Common attributes
      • Filters
        • By type
        • By expression
        • Send on delta
        • Remove attributes
        • Drop all events
      • Enrichment
        • Key concepts
          • Anatomy of enrichment DSL
          • Banking example
        • Metrics
        • Dynamic contextual data
          • Caching architecture
        • Static contextual data
      • Transformation
        • Field Tokeniser
        • Obfuscation
          • Encryption
          • Masking
          • Bucketing
          • Redaction
      • Triggers
        • Change Data Capture
        • Business rules
      • Stream join
        • Inner stream joins
        • Outer stream joins
        • Join attributes & policy
      • Event tap
        • Anatomy of a Tap
        • SQL Queries
    • Analytics
      • Analytic tools
        • User defined analytics
          • Streaming analytics example
          • User defined analytics
          • User defined scripts
          • User defined functions
            • Average function library
        • Window analytics
          • Tumbling window
          • Sliding window
          • Aggregate functions
        • Analytic functions
          • Stateful
            • Exponential moving average
            • Rolling Sum
          • Stateless
            • Normalisation
              • Absolute max
              • Min max
              • Standardisation
              • Mean
              • Log
              • Z-Score
            • Scaling
              • Unit scale
              • Robust Scale
            • Statistics
              • Statistic summaries
              • Weighted moving average
              • Simple moving average
              • Count
            • General
              • Euclidean
        • Advanced analytics
          • Geospatial
            • Entity geo tracker
            • Geofence occupancy trigger
            • Geo search
            • IP address resolver
            • Reverse geocoding
            • Spatial Index
          • HyperLogLog
          • Distinct counter
      • ML inferencing
        • Feature engineering
          • Scripting
          • Scaling
          • Transform
        • Online predictive analytics
        • Model audit
        • Model management
      • Metrics engine
        • Create metrics
        • Apply metrics
        • Manage metrics
        • Priming metrics
    • Contextual data
      • Architecture
      • Configuration
      • MinIO S3
      • Apache Geode
    • Connectors
      • Sources
        • Kafka
          • Ingestion
        • RabbitMQ
          • Further RabbitMQ configurations
        • MQTT
          • Topic wildcards
          • Session management
          • Last Will and Testament
        • Rest endpoints
        • MinIO S3
        • File watcher
      • Sinks
        • Kafka
        • RabbitMQ
          • Further configurations
        • MQTT
          • Persistent messaging
          • Last Will and Testament
        • SQL databases
        • InfluxDB
        • MongoDB
        • Geode
        • WebSocket endpoint
        • MinIO S3
        • File transport
        • Slack
        • Email
      • Serialisers
        • Serialisation
          • Custom transform example
          • Formatters
        • Deserialisers
          • Custom parsing example
    • Observability
      • Enabling JMX for Joule
      • Meters
      • Metrics API
  • DEVELOPER GUIDES
    • Setting up developer environment
      • Environment setup
      • Build and deploy
      • Install Joule
        • Install Docker demo environment
        • Install with Docker
        • Install from source
        • Install Joule examples
    • Joulectl CLI
    • API Endpoints
      • Mangement API
        • Use case
        • Pipelines
        • Data connectors
        • Contextual data
      • Data access API
        • Query
        • Upload
        • WebSocket
      • SQL support
    • Builder SDK
      • Connector API
        • Sources
          • StreamEventParser API
        • Sinks
          • CustomTransformer API
      • Processor API
      • Analytics API
        • Create custom metrics
        • Define analytics
        • Windows API
        • SQL queries
      • Transformation API
        • Obfuscation API
        • FieldTokenizer API
      • File processing
      • Data types
        • StreamEvent
        • ReferenceDataObject
        • GeoNode
    • System configuration
      • System properties
  • Deployment strategies
    • Deployment Overview
    • Single Node
    • Cluster
    • GuardianDB
    • Packaging
      • Containers
      • Bare metal
  • Product updates
    • Public Roadmap
    • Release Notes
      • v1.2.0 Join Streams with stateful analytics
      • v1.1.0 Streaming analytics enhancements
      • v1.0.4 Predictive stream processing
      • v1.0.3 Contextual SQL based metrics
    • Change history
Powered by GitBook
On this page
  • Objective
  • Key Features
  • Example & DSL attributes
  • Attributes schema
  • Further example
  • Example function implementation

Was this helpful?

  1. Components
  2. Analytics
  3. Analytic tools
  4. User defined analytics

User defined functions

Extend the analytics ecosystem using the Analytics API

PreviousUser defined scriptsNextAverage function library

Last updated 4 months ago

Was this helpful?

Objective

This analytic processor enables complex calculation to be executed using a provided analytical function implemented using the . This feature also provides the ability to set a rolling number of events to be used with the calculation.

Leverage this feature for complex calculations and to control the implementation

Learn different average functions which can be applied in Joule.

Key Features

  • Pluggable

  • Analytics API

  • DSL support

Example & DSL attributes

The analytic function ema (Exponential Moving Average) is defined inline using the standard Joule DSL fragment.

user defined function:
  function:
    exponential moving average:
      parameters:
        smoothing factor: 0.33333
  fields: [ ask ]
  event history: 12
  assign prefix: ema12

Attributes schema

Attribute
Description
Data Type
Required

fields

Fields to calculated from the event

String[]

event history

Number of rolling events to stored and used for the calculation

Integer

assign prefix

Prefix to use for the result assignment. This is used to allow the same function to be used multiple times. If this is not provided the function postfix will be applied e.g. ask_EMA

String Default: Function postfix

Further example

This example demonstrates how you would combine complex calculation with a final analytic expression to trigger an alert.

processing unit:
    pipeline:
      - user defined function:
          function:
            exponential moving average:
                parameters:
                  smoothing factor: 0.33333
          fields: [ ask ]
          event history: 26
          response prefix: ema26
    
      - user defined function:
          function:
            exponential moving average:
                parameters:
                  smoothing factor: 0.33333
          fields: [ ask ]
          event history: 12
          response prefix: ema12
    
      - analytic:
          expression: "ema12_ask - ema26_ask"
          assign to: macd_ask_signal
      
emit:
    select: "symbol, macd_ask_signal"
    having: "macd_ask_signal > 0.05"

Example function implementation

This is an implementation example for a custom analytics function.

@JsonRootName(value = "exponential moving average")
public class ExponentialMovingAverage extends AnalyticsFunction<Double> {

    public static final String SMOOTH_FACTOR = "smoothing factor";
    private double smoothFactor = 0.333;

    public ExponentialMovingAverage() {
        super();
    }

    @Override
    public void setParameters(Properties parameters) {
        if (parameters != null && parameters.containsKey(SMOOTH_FACTOR)) {
            smoothFactor = Double.parseDouble(parameters.get(SMOOTH_FACTOR).toString());
        }
    }

    @Override
    public String getVariablePostFixID() {
        return "EMA";
    }

    @Override
    public Double compute(Number[] values, Number previousValue) {
        if (previousValue == null) return Double.NaN;

        if (Double.isNaN(previousValue.doubleValue())) {
            previousValue = values[values.length - 1];
            return (Double)previousValue;
        }

        for (Number d : values) {
            previousValue = (d.doubleValue() * smoothFactor) + (previousValue.doubleValue() * (1 - smoothFactor));
        }
        return previousValue.doubleValue();
    }
}

Further details can be found in the documentation.

Analytics API
Average function library
Analytics API