Joule
  • Welcome to Joule's Docs
  • Why Joule?
    • Joule capabilities
  • What is Joule?
    • Key features
    • The tech stack
  • Use case enablement
    • Use case building framework
  • Concepts
    • Core concepts
    • Low code development
    • Unified execution engine
    • Batch and stream processing
    • Continuous metrics
    • Key Joule data types
      • StreamEvent object
      • Contextual data
      • GeoNode
  • Tutorials
    • Getting started
    • Build your first use case
    • Stream sliding window quote analytics
    • Advanced tutorials
      • Custom missing value processor
      • Stateless Bollinger band analytics
      • IoT device control
  • FAQ
  • Glossary
  • Components
    • Pipelines
      • Use case anatomy
      • Data priming
        • Types of import
      • Processing unit
      • Group by
      • Emit computed events
      • Telemetry auditing
    • Processors
      • Common attributes
      • Filters
        • By type
        • By expression
        • Send on delta
        • Remove attributes
        • Drop all events
      • Enrichment
        • Key concepts
          • Anatomy of enrichment DSL
          • Banking example
        • Metrics
        • Dynamic contextual data
          • Caching architecture
        • Static contextual data
      • Transformation
        • Field Tokeniser
        • Obfuscation
          • Encryption
          • Masking
          • Bucketing
          • Redaction
      • Triggers
        • Change Data Capture
        • Business rules
      • Stream join
        • Inner stream joins
        • Outer stream joins
        • Join attributes & policy
      • Event tap
        • Anatomy of a Tap
        • SQL Queries
    • Analytics
      • Analytic tools
        • User defined analytics
          • Streaming analytics example
          • User defined analytics
          • User defined scripts
          • User defined functions
            • Average function library
        • Window analytics
          • Tumbling window
          • Sliding window
          • Aggregate functions
        • Analytic functions
          • Stateful
            • Exponential moving average
            • Rolling Sum
          • Stateless
            • Normalisation
              • Absolute max
              • Min max
              • Standardisation
              • Mean
              • Log
              • Z-Score
            • Scaling
              • Unit scale
              • Robust Scale
            • Statistics
              • Statistic summaries
              • Weighted moving average
              • Simple moving average
              • Count
            • General
              • Euclidean
        • Advanced analytics
          • Geospatial
            • Entity geo tracker
            • Geofence occupancy trigger
            • Geo search
            • IP address resolver
            • Reverse geocoding
            • Spatial Index
          • HyperLogLog
          • Distinct counter
      • ML inferencing
        • Feature engineering
          • Scripting
          • Scaling
          • Transform
        • Online predictive analytics
        • Model audit
        • Model management
      • Metrics engine
        • Create metrics
        • Apply metrics
        • Manage metrics
        • Priming metrics
    • Contextual data
      • Architecture
      • Configuration
      • MinIO S3
      • Apache Geode
    • Connectors
      • Sources
        • Kafka
          • Ingestion
        • RabbitMQ
          • Further RabbitMQ configurations
        • MQTT
          • Topic wildcards
          • Session management
          • Last Will and Testament
        • Rest endpoints
        • MinIO S3
        • File watcher
      • Sinks
        • Kafka
        • RabbitMQ
          • Further configurations
        • MQTT
          • Persistent messaging
          • Last Will and Testament
        • SQL databases
        • InfluxDB
        • MongoDB
        • Geode
        • WebSocket endpoint
        • MinIO S3
        • File transport
        • Slack
        • Email
      • Serialisers
        • Serialisation
          • Custom transform example
          • Formatters
        • Deserialisers
          • Custom parsing example
    • Observability
      • Enabling JMX for Joule
      • Meters
      • Metrics API
  • DEVELOPER GUIDES
    • Setting up developer environment
      • Environment setup
      • Build and deploy
      • Install Joule
        • Install Docker demo environment
        • Install with Docker
        • Install from source
        • Install Joule examples
    • Joulectl CLI
    • API Endpoints
      • Mangement API
        • Use case
        • Pipelines
        • Data connectors
        • Contextual data
      • Data access API
        • Query
        • Upload
        • WebSocket
      • SQL support
    • Builder SDK
      • Connector API
        • Sources
          • StreamEventParser API
        • Sinks
          • CustomTransformer API
      • Processor API
      • Analytics API
        • Create custom metrics
        • Define analytics
        • Windows API
        • SQL queries
      • Transformation API
        • Obfuscation API
        • FieldTokenizer API
      • File processing
      • Data types
        • StreamEvent
        • ReferenceDataObject
        • GeoNode
    • System configuration
      • System properties
  • Deployment strategies
    • Deployment Overview
    • Single Node
    • Cluster
    • GuardianDB
    • Packaging
      • Containers
      • Bare metal
  • Product updates
    • Public Roadmap
    • Release Notes
      • v1.2.0 Join Streams with stateful analytics
      • v1.1.0 Streaming analytics enhancements
      • v1.0.4 Predictive stream processing
      • v1.0.3 Contextual SQL based metrics
    • Change history
Powered by GitBook
On this page
  • What will we learn on this article?
  • Core Joule architecture
  • Event sources
  • Stream processors
  • Data sinks
  • Joule Domain Specific Language
  • Internal event representation

Was this helpful?

  1. Concepts

Core concepts

This section explores high-level concepts about Joule, it will help us understand how to build stream processing pipelines that subscribe to source data and publish results

PreviousConceptsNextLow code development

Last updated 5 months ago

Was this helpful?

What will we learn on this article?

This article introduces high-level concepts about Joule, providing a foundational understanding of key topics and how they connect.

You will learn how Joule enables the creation of stream processing pipelines that subscribe to source data and publish results.

This article covers:

  • Joule core concepts An overview of essential concepts necessary for using Joule effectively.

  • Event sources Discover how data sources supply Joule with the information needed to execute use cases, including implementations like Kafka, RabbitMQ and others.

  • Stream processors Understand the role of processors in Joule, which perform unique functions that, when combined, create business use cases. Examples include enrichers, transformations and online predictions.

  • Data sinks Learn how data sinks allow Joule to connect with downstream systems, adding value to business cases through implementations like SQL databases and InfluxDB.

  • Joule Domain Specific Language (DSL) Explore how Joule's DSL facilitates defining use cases with human-readable YAML syntax.

  • Internal event representation Understand the key internal event structure to develop use cases and extend the platform.

By the end of this article, you will have a comprehensive understanding of Joule's capabilities and components.

Learn more about the

Core Joule architecture

Stream & batch processed data can be combined with Joule.

Event sources

Data sources provide Joule with the required data to execute use cases.

Fractalworks has provided a number of ready to use implementations to enable you to start building use cases.

Stream processors

Processors form the core of the Joule platform.

A processor performs a distinct unique function when chained with others form a business use case.

Data sinks

Data sinks enable Joule to integrate to downstream systems which provide further value to additional business cases

Joule Domain Specific Language

The Joule platform provides a Domain Specific Language (DSL) to define use cases using human-readable yaml syntax.

Internal event representation

When data is processed within Joule external data is converted in to an internal format before being presented to consumers in to a suitable format.

Sources
Processors
Sinks
Low code development
StreamEvent
core data types of Joule on this pages
Core concept architecture