Sources
Connect to key data sources to drive use cases
Overview
Sources provide a flexible data processing framework, enabling users to quickly set up and run use cases by transforming raw data into a structured format suitable for pipeline processing.
With built-in support for multiple data ingestion methods, Joule allows you to efficiently bring in data from various sources, converting it into StreamEvent objects that flow through the platform until the publishing stage.
Fractalworks has also developed ready-to-use connectors to speed up development.
Build your own connectors using the Connector API and quick start templates
Available connector options
Incoming events are deserialised into StreamEvents objects for pipeline processing. This can be done automatically or via a custom transformer, depending on your needs. Supported ingestion methods include:
Streaming
Kafka
Standard Kafka consumers transport ingests data from subscribed cluster topics
Rabbit MQ
RabbitMQ is a lightweight and easy to deploy messaging platform
MQTT
Joule provides the ability to subscribe to events using MQTT source consumer
HTTP
File
Last updated