Serialisers

Get data ready for consumption serialising incoming and outgoing data

Overview

Incoming and outgoing data requires mapping to appropriate formats to support seamless processing and distribution within Joule. Serialisers facilitate this process by efficiently translating between source and target data types.

In computing, serialization (or serialisation) is the process of translating a data structure or object state into a format that can be stored (e.g. files in secondary storage devices, data buffers in primary storage devices) or transmitted (e.g. data streams over computer networks) and reconstructed later (possibly in a different computer environment).

wikipedia

Joule processes incoming data by converting it from the source format into a StreamEvent.

StreamEvents can be transformed back into domain-specific data types, making them ready for consumption by other systems.

Supported tools

1. Developer SDK Ability to write, deploy and reuse custom domain parsers and transformers.

2. AVRO support Support to read and write domain-specific data types.

3. Native StreamEvent Binary and JSON StreamEvent interoperability support between Joule processes.

Available serialising options

Serialisation

Data emitted to downstream systems is serialised using provided or custom serialisers

Deserialisation

Consumed data is mapped to internal StreamEvents using provided or custom deserialisers