Connectors

Out-of-the-box Joule provides key data connectors for streaming use cases

Overview

Data connectors, sources and sinks are a key component within the Joule low-code ecosystem that consume and distribute data.

To get you started processing data Joule provides a catalog of widely used standard connector solutions. For example, Joule supports real-time data streaming and batch processing connector integrations with major industry standards such as Kafka, MQTT and RabbitMQ, facilitating seamless data transfer across diverse applications and systems.

If any of the provided connectors do not fit your requirements, Joule makes it possible to build your own connector by leveraging the Connector API. This SDK offers customisable templates and a robust API, empowering business developers to create connectors tailored to their unique data sources or destinations.

Key features of Joule's connectors capability include

1. Streaming Standard industry messaging connectors such as Kafka, MQTT and RabbitMQ are provided for real-time use cases.

2. Batch processing Joule does not differentiate between batch and real-time data; All incoming batch data is converted to stream events and outgoing can be either batched or streamed.

3. Additive connector features Where ever possible additional features have been added to connectors such as wire compression, auto data type translations, change listeners etc.

4. Flexible transformation Ingesting and publishing requires some level of data transformation complexity. Standard formats such as JSON, CSV, Parquet, AVRO and the ability to define custom transformer, (de)serialisation code are all supported.

Types of connectors & serialisation

Build your own connectors using the Connector API and quick start templates

Sources

Connect to key data sources to drive use cases

Sinks

Distribute insights to consuming systems

Serialisers

Get data ready for consumption serialising incoming and outgoing data