Core concepts

This section explores high-level concepts about Joule, it will help us understand how to build stream processing pipelines that subscribe to source data and publish results

What will we learn on this article?

This article introduces high-level concepts about Joule, providing a foundational understanding of key topics and how they connect.

You will learn how Joule enables the creation of stream processing pipelines that subscribe to source data and publish results.

This article covers:

• Joule core concepts An overview of essential concepts necessary for using Joule effectively.

• Event sources Discover how data sources supply Joule with the information needed to execute use cases, including implementations like Kafka, RabbitMQ and others.

• Stream processors Understand the role of processors in Joule, which perform unique functions that, when combined, create business use cases. Examples include enrichers, transformations and online predictions.

• Data sinks Learn how data sinks allow Joule to connect with downstream systems, adding value to business cases through implementations like SQL databases and InfluxDB.

• Joule Domain Specific Language (DSL) Explore how Joule's DSL facilitates defining use cases with human-readable YAML syntax.

• Internal event representation Understand the key internal event structure to develop use cases and extend the platform.

By the end of this article, you will have a comprehensive understanding of Joule's capabilities and components.

Learn more about the core data types of Joule on this pages

Core Joule architecture

Core concept architecture

Stream & batch processed data can be combined with Joule.

Event sources

Data sources provide Joule with the required data to execute use cases.

Fractalworks has provided a number of ready to use implementations to enable you to start building use cases.

Sources

Stream processors

Processors form the core of the Joule platform.

A processor performs a distinct unique function when chained with others form a business use case.

Processors

Data sinks

Data sinks enable Joule to integrate to downstream systems which provide further value to additional business cases

Sinks

Joule Domain Specific Language

The Joule platform provides a Domain Specific Language (DSL) to define use cases using human-readable yaml syntax.

Low code development

Internal event representation

When data is processed within Joule external data is converted in to an internal format before being presented to consumers in to a suitable format.

StreamEvent