v1.1.0 Streaming analytics enhancements
Build streaming analytics pipelines powered by machine learning, metrics and reference data using live data
Last updated
Build streaming analytics pipelines powered by machine learning, metrics and reference data using live data
Last updated
Version 1.1.0
Joule’s latest release offers businesses a comprehensive solution to accelerate use case development to generate value while minimising risk. The platform leverages dynamic ML models, metrics, reference data, and observability to provide real-time actions and insights.
With Joule, businesses can streamline their development efforts and make informed decisions based on data-driven insights. Joule’s intuitive development platform and user-focused design make it easy for businesses to leverage the power of data and maximise their potential.
JPMML model initialisation using local file and remote S3 stores
Dynamic model refresh using model update notifications
Offline prediction auditing that enables explainability, drift monitoring and model retraining
Ability to process avro records for inbound and outbound events
Complex data types supported using custom mapping
Schema registry support
OOTB multi cloud S3 support
Publish and consume events and insights to/from hybrid hosted S3 buckets
Drive pipeline processing using S3 bucket notifications
Consumer supports following file formats: PARQUET, CSV, ARROW, ORC
Keep reference data up to date using external systems
Apply external data within stream processing tasks
In-memory reference data elements kept up-to-date using source change notifications
Support for key value and S3 stores
Reference data file loader utility
File consuming endpoint that enable ease of integration to upstream systems
Joule event consumer endpoint to provide the ability to chain Joule processors within a cloud environment
File watcher that consumes and processes target files
Supported formats; Parquet, Json, CSV, ORC and Arrow IPC
Confluent schema registry support for outbound events
Message partition support
Confluent and RedPanda support
Query optimisation
SQL, OQL, and Key value enrichment support
Improved exception handling to fail on startup
Strict ordering
Integrated and leveraged to process file efficiently and of various file formats
Large file processing support
Processing optimisations that reduce both memory and CPU utilisation while increasing event throughput.
StreamEvent smart shallow cloning logic to reduce overall memory footprint while providing key data isolation
StreamEvent change tracking switch to reduce memory overhead
Javalin 5.6.3
Kafka 3.6.0
Avro 1.11.3
DuckDB 0.9.2
StreamEventCSVDeserializer
Fixed fields from holding only string values to correctly defined data types
Allowed for custom date format to be provided
StreamEventJSONDeserializer
Can now read an array of Json StreamEvent objects
Require ‘--add-opens=java.base/java.nio=ALL-UNNAMED’ to be added to the java CLI due to Apache Arrow requirements
Applying the G1 GC regionalized and generational garbage collector to improved memory usage
