Joule
  • Welcome to Joule's Docs
  • Why Joule?
    • Joule capabilities
  • What is Joule?
    • Key features
    • The tech stack
  • Use case enablement
    • Use case building framework
  • Concepts
    • Core concepts
    • Low code development
    • Unified execution engine
    • Batch and stream processing
    • Continuous metrics
    • Key Joule data types
      • StreamEvent object
      • Contextual data
      • GeoNode
  • Tutorials
    • Getting started
    • Build your first use case
    • Stream sliding window quote analytics
    • Advanced tutorials
      • Custom missing value processor
      • Stateless Bollinger band analytics
      • IoT device control
  • FAQ
  • Glossary
  • Components
    • Pipelines
      • Use case anatomy
      • Data priming
        • Types of import
      • Processing unit
      • Group by
      • Emit computed events
      • Telemetry auditing
    • Processors
      • Common attributes
      • Filters
        • By type
        • By expression
        • Send on delta
        • Remove attributes
        • Drop all events
      • Enrichment
        • Key concepts
          • Anatomy of enrichment DSL
          • Banking example
        • Metrics
        • Dynamic contextual data
          • Caching architecture
        • Static contextual data
      • Transformation
        • Field Tokeniser
        • Obfuscation
          • Encryption
          • Masking
          • Bucketing
          • Redaction
      • Triggers
        • Change Data Capture
        • Business rules
      • Stream join
        • Inner stream joins
        • Outer stream joins
        • Join attributes & policy
      • Event tap
        • Anatomy of a Tap
        • SQL Queries
    • Analytics
      • Analytic tools
        • User defined analytics
          • Streaming analytics example
          • User defined analytics
          • User defined scripts
          • User defined functions
            • Average function library
        • Window analytics
          • Tumbling window
          • Sliding window
          • Aggregate functions
        • Analytic functions
          • Stateful
            • Exponential moving average
            • Rolling Sum
          • Stateless
            • Normalisation
              • Absolute max
              • Min max
              • Standardisation
              • Mean
              • Log
              • Z-Score
            • Scaling
              • Unit scale
              • Robust Scale
            • Statistics
              • Statistic summaries
              • Weighted moving average
              • Simple moving average
              • Count
            • General
              • Euclidean
        • Advanced analytics
          • Geospatial
            • Entity geo tracker
            • Geofence occupancy trigger
            • Geo search
            • IP address resolver
            • Reverse geocoding
            • Spatial Index
          • HyperLogLog
          • Distinct counter
      • ML inferencing
        • Feature engineering
          • Scripting
          • Scaling
          • Transform
        • Online predictive analytics
        • Model audit
        • Model management
      • Metrics engine
        • Create metrics
        • Apply metrics
        • Manage metrics
        • Priming metrics
    • Contextual data
      • Architecture
      • Configuration
      • MinIO S3
      • Apache Geode
    • Connectors
      • Sources
        • Kafka
          • Ingestion
        • RabbitMQ
          • Further RabbitMQ configurations
        • MQTT
          • Topic wildcards
          • Session management
          • Last Will and Testament
        • Rest endpoints
        • MinIO S3
        • File watcher
      • Sinks
        • Kafka
        • RabbitMQ
          • Further configurations
        • MQTT
          • Persistent messaging
          • Last Will and Testament
        • SQL databases
        • InfluxDB
        • MongoDB
        • Geode
        • WebSocket endpoint
        • MinIO S3
        • File transport
        • Slack
        • Email
      • Serialisers
        • Serialisation
          • Custom transform example
          • Formatters
        • Deserialisers
          • Custom parsing example
    • Observability
      • Enabling JMX for Joule
      • Meters
      • Metrics API
  • DEVELOPER GUIDES
    • Setting up developer environment
      • Environment setup
      • Build and deploy
      • Install Joule
        • Install Docker demo environment
        • Install with Docker
        • Install from source
        • Install Joule examples
    • Joulectl CLI
    • API Endpoints
      • Mangement API
        • Use case
        • Pipelines
        • Data connectors
        • Contextual data
      • Data access API
        • Query
        • Upload
        • WebSocket
      • SQL support
    • Builder SDK
      • Connector API
        • Sources
          • StreamEventParser API
        • Sinks
          • CustomTransformer API
      • Processor API
      • Analytics API
        • Create custom metrics
        • Define analytics
        • Windows API
        • SQL queries
      • Transformation API
        • Obfuscation API
        • FieldTokenizer API
      • File processing
      • Data types
        • StreamEvent
        • ReferenceDataObject
        • GeoNode
    • System configuration
      • System properties
  • Deployment strategies
    • Deployment Overview
    • Single Node
    • Cluster
    • GuardianDB
    • Packaging
      • Containers
      • Bare metal
  • Product updates
    • Public Roadmap
    • Release Notes
      • v1.2.0 Join Streams with stateful analytics
      • v1.1.0 Streaming analytics enhancements
      • v1.0.4 Predictive stream processing
      • v1.0.3 Contextual SQL based metrics
    • Change history
Powered by GitBook
On this page

Was this helpful?

  1. Product updates

Change history

Previousv1.0.3 Contextual SQL based metrics

Last updated 20 days ago

Was this helpful?

Change log

Version
Date
Description

1.2.1-SNAPSHOT

11-May-2025

Features

Joule cluster

  • Joule cluster deployment support

  • Ability to run Joule use cases across multiple nodes

Joulectl CLI

  • Command line tool to deploy and manage Joule

Dynamic Deployment

  • Suite of Joule management Rest APIs for use case deployment

  • Joule can run in a standalone state or as a daemon

  • Ability to restart using previous deployed state

  • Joule management database (GuardianDB)

  • Basic process telemetry recording

Transports

  • New slack transport for proactive alerting

  • New email transport for critiacal notifications

General

  • Applied processing time constraints to use case execution

  • Caching of last emitted event for next processing cycle reuse

  • General state management to support advanced processing use cases

  • New SDK processor and transport annotations

Getting started project

  • New refreshed suite of demos to addressed fast tracking developers on to the platform

Dependencies

  • Upgraded following

    • DuckDB 1.1.3

    • Jackson 2.18.1

    • Apache arrow 18.1.0

Fixes and improvements

  • New and improved documentation

  • Fixed duplicate events being emitted from an analytic window processing

  • Optimised and improved AVRO record handling

  • Improved Kafka commit and ack logic

  • Fixed missing CSV header

  • Projection order honoured for CSV file output

  • OpenAPI documentation improvements

  • Memory optimisations and handling improvements

  • Increased default internal base resources

  • Support for key data types for stateful analytics

  • Fixed major null bug for object vectorization

  • Fixed MQTT missing initialization and serialization of string payloads

  • Platform build target moved to JDK 23

  • Upgrade Gradle 8.12

1.2.0

01-July

Features

Stream Joins

  • Ability to join two streams using a simple join expression

  • Inner and outer joins support

  • State managed either through an time to live function or simply delete on join

Enrichment

  • Computed metrics can be added to events using the enrichment processor and emit projection

  • Internal in-memory database can be queried for event enrichment

  • Stateful computation support

Metrics

  • Import pre-computed metrics on process startup

  • User defined metrics compaction query

Enhancements

Kafka

  • Validated consumer message partition examples using multi Joule processing

Analytics

  • New normalization, scaling, and statistics functions added for ML support

  • Stateful computation applied to computation context

  • Simplified DSL to enable correct processor to be used

  • Improvements and bug fixes to address execution of expressions and scripts

  • Documentation improvements

Miscellaneous

  • Multi Joule processing environment scripts, used within banking examples

  • groupby definition support across all processors

  • Javascript module mjs support

Upgrades

  • DuckDB 1.0.0

1.1.0

25-Jan-2024

Features

Predictive Processor

  • JPMML model initialization using local file and remote S3 stores

  • Dynamic model refresh using model update notifications

  • Offline prediction auditing that enables explainability, drift monitoring and model retraining

Avro support

  • Ability to process avro records for inbound and outbound events

  • Complex data types supported using custom mapping

  • Schema registry support

Minio S3 Transport

  • OOTB multi cloud S3 support

  • Publish and consume events and insights to/from hybrid hosted S3 buckets

  • Drive pipeline processing using S3 bucket notifications

  • Consumer supports following file formats: PARQUET, CSV, ARROW, ORC

  • Keep reference data up to date using external systems

Reference Data

  • Apply external data within stream processing tasks

  • In-memory reference data elements kept up-to-date using source change notifications

  • Support for key value and S3 stores

  • Reference data file loader utility

Rest Consumer APIs

  • File consuming endpoint that enable ease of integration to upstream systems

  • Joule event consumer endpoint to provide the ability to chain Joule processors within a cloud environment

File Watcher Consumer

  • File watcher that consumes and processes target files

  • Supported formats; Parquet, Json, CSV, ORC and Arrow IPC

Enhancements

Kafka

  • Confluent schema registry support for outbound events

  • Message partition support

  • Confluent and RedPanda support

Enricher processor

  • Query optimisation

  • SQL, OQL, and Key value enrichment support

Transports

  • Improved exception handling to fail on startup

  • Strict ordering

Apache Arrow

  • Integrated and leveraged to process file efficiently and of various file formats

  • Large file processing support

Optimisations

  • Processing optimisations that reduce both memory and CPU utilisation while increasing event throughput.

  • StreamEvent smart shallow cloning logic to reduce overall memory footprint while providing key data isolation

  • StreamEvent change tracking switch to reduce memory overhead

Upgrades

  • Javalin 5.6.3

  • Kafka 3.6.0

  • Avro 1.11.3

  • DuckDB 0.9.2

Bug Fixes

StreamEventCSVDeserializer

  • Fixed fields from holding only string values to correctly defined data types

  • Allowed for custom date format to be provided

StreamEventJSONDeserializer

  • Can now read an array of Json StreamEvent objects

JVM Configuration Additions

  • Require ‘--add-opens=java.base/java.nio=ALL-UNNAMED’ to be added to the java CLI due to Apache Arrow requirements

  • Applying the G1 GC regionalized and generational garbage collector to improved memory usage

1.1.0-SNAPSHOT

14-Sept 2023

Features

  • Predictive Processor Model initialisation and dynamic model replacement using local file and remote S3 stores

  • Avro support for inbound and outbound events.

  • Complex data types supported using custom field mapping

  • Minio multi-cloud S3 consumer

  • Reference data support

Enhancements

  • MinIO implementation improvements to support large files and notifications

  • Kafka schema registry support

  • Apache Arrow integration to support PARQUET, CSV, ARROW, ORC file formats

  • Better transport exception handling

1.0.4

18-May-2023

Features

  • Feature engineering processor, Including plugins for binning, scaling and declarative analytics expressions

  • Prediction and features auditing enabling explainability, drift monitoring and retraining

  • In-Memory SQL Query API

  • Websocket publisher

Enhancements

  • Improved deep cloning of StreamEvent

  • Reorganisation of SDK

  • Documentation

  • Project template

Bug fixes

  • Window events failing to publish

1.0.4-SNAPSHOT

23-Apr-2023

Docker image generation, scripting fixes, StreamEvent deep copy, documentation updates.

1.0.3

01-Mar-2023

Documentation, metrics engine, Rest API, JDBC publisher, transport and processor updates and bug fixes

1.0.2

30-Jan-2023

Alpha release of metrics engine and rest API

1.0.1

13-May-2022

Bug fixes

1.0.0

05-Apr-2022

Initial release

Java 23 Graalvm