Data connectors

Joule provided key data connectors out-of-the-box for stream use cases

Overview

Data connectors are a key component within the Joule low-code ecosystem to send data in and out of Joule.

In other words, connectors either consume or distribute data. This is also known as sources and sinks.

Joule provides a catalog of connectors to get you started out-of-the-box, using widely used connector solutions.

In case any of the out-of-the-box connectors do not fit your requirements, it is possible to build your own connector leveraging the Connector SDK.

Key features of Joule's connectors capability include

1. Streaming Standard industry messaging connectors such as Kafka, MQTT and RabbitMQ are provided for real-time use cases.

2. Batch processing Joule does not differentiate between batch and real-time data; All incoming batch data is converted to stream events and outgoing can be either batched or streamed.

3. Additive connector features Where ever possible additional features have been added to connectors such as wire compression, auto data type translations, change listeners etc;.

4. Flexible transformation Ingesting and publishing requires some level of data transformation complexity. Standard formats such as Json, CSV, Parquet, AVRO and ability to define custom transformer, serialization, code are all supported.

Types of connectors

Sources

Connect to key data sources to drive use cases

Sinks

Distribute insights to consuming systems

Serialization

Data ingestion and publishing transformation is performed by serialization.

Serialization

Incoming and outgoing data require mapping to suitable data formats ready for consumption and processing

Extending capabilities

Joule provides a Connector SDK to enable business developers to extend the data sourcing and distribution capabilities.

Build your own connectors using the Connector API and quick start templates