StreamEvent
An event drives all processing. At its core Joule defines a StreamEvent that is passed from one component to another until it is ejected to consumers.
Overview
External data events are converted into an internal data type, StreamEvent
, to enable a flexible standard processor model to occur.
The StreamEvent provides the vehicle to transport and process data throughout the platform
Understanding this structure and how to use it enables you to develop custom components that are optimised for processing efficiency while providing the enough flexibility to represent external data within the Joule platform.
Key Features
Typed by source system
Timestamped by event occurrence at source or at time of platform ingestion
Changes trackable by time, value and by affecting processor
Cloneable, deep copy, to preserve transaction isolation within processing context
Serializable for external consumption
StreamEvent can be located under the following package
Attributes
Attribute | Description | Type |
---|---|---|
uuid | Unique identifier assigned by the platform to enable tracking | UUID |
type | Parent type of the event e.g. market_data | String |
subType | Finer grain type of the event e.g. aim | Object |
eventTme | Actual time the event occured within the source system. This is set by the user. | Long |
IngestTime | Actual time the event was ingested by Joule process. This is automatically assigned at point of object creation. | Long |
dictionary | Store of all key event attributes. | Map<String,Object> |
callChain | Tracking store of every change made to the event as it passes through the processing pipeline. This is managed by the Joule platform. | Map<Tuple<UUID, String>, Long> |
Key Methods
Adding
To add new attributes to an event there are two methods which can be used. If a null value has been passed this will be converted to a String null
value.
Updating
Attributes can be updated in place. If a null value has been passed this will be converted to a String null
value.
Get value
Either a single attribute or a the whole set of attributes values be returned to the caller
Remove
Tow methods exists to remove fields from the event.
Cloning
A deep copy method is provided to return a new StreamEvent from the event.
Get call chain
To get the history of changes applied to the event use the below method.
Last updated