# v1.1.0 Streaming analytics enhancements

{% hint style="success" %}
Version 1.1.0
{% endhint %}

## Overview

Joule’s latest release offers businesses a comprehensive solution to accelerate use case development to generate value while minimising risk. The platform leverages dynamic ML models, metrics, reference data, and observability to provide real-time actions and insights.<br>

With Joule, businesses can streamline their development efforts and make informed decisions based on data-driven insights. Joule’s intuitive development platform and user-focused design make it easy for businesses to leverage the power of data and maximise their potential.

<figure><img src="/files/hw5zxoj1waDSpHcLenGi" alt=""><figcaption></figcaption></figure>

## Features

### Predictive Processor

* JPMML model initialisation using local file and remote S3 stores
* Dynamic model refresh using model update notifications
* Offline prediction auditing that enables explainability, drift monitoring and model retraining

### Avro support&#x20;

* Ability to process avro records for inbound and outbound events
* Complex data types supported using custom mapping
* Schema registry support

### Minio S3 Transport

* OOTB multi cloud S3 support
* Publish and consume events and insights to/from hybrid hosted S3 buckets&#x20;
* Drive pipeline processing using S3 bucket notifications
* Consumer supports following file formats: PARQUET, CSV, ARROW, ORC
* Keep reference data up to date using external systems

### Reference Data

* Apply external data within stream processing tasks
* In-memory reference data elements kept up-to-date using source change notifications&#x20;
* Support for key value and S3 stores
* Reference data file loader utility

### Rest Consumer APIs

* File consuming endpoint that enable ease of integration to upstream systems
* Joule event consumer endpoint to provide the ability to chain Joule processors within a cloud environment

### File Watcher Consumer

* File watcher that consumes and processes target files
* Supported formats; Parquet, Json, CSV, ORC and Arrow IPC

***

## Enhancements

### Kafka

* Confluent schema registry support for outbound events
* Message partition support
* Confluent and RedPanda support

### Enricher processor

* Query optimisation
* SQL, OQL, and Key value enrichment support

### Transports

* Improved exception handling to fail on startup
* Strict ordering

### Apache Arrow

* Integrated and leveraged to process file efficiently and of various file formats
* Large file processing support

***

## Optimisations&#x20;

* Processing optimisations that reduce both memory and CPU utilisation while increasing event throughput.
* StreamEvent smart shallow cloning logic to reduce overall memory footprint while providing key data isolation
* StreamEvent change tracking switch to reduce memory overhead

## Upgrades

* Javalin 5.6.3
* Kafka 3.6.0
* Avro 1.11.3
* DuckDB 0.9.2

## Bug Fixes

StreamEventCSVDeserializer

* Fixed fields from holding only string values to correctly defined data types
* Allowed for custom date format to be provided&#x20;

StreamEventJSONDeserializer

* Can now read an array of Json StreamEvent objects

### JVM Configuration Additions

* Require  ‘--add-opens=java.base/java.nio=ALL-UNNAMED’ to be added to the java CLI due to Apache Arrow requirements
* Applying the G1 GC regionalized and generational garbage collector to improved memory usage


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.fractalworks.io/joule/product-updates/release-notes/v1.1.0-streaming-analytics-enhancements.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
