Joule
  • Welcome to Joule's Docs
  • Why Joule?
    • Joule capabilities
  • What is Joule?
    • Key features
    • The tech stack
  • Use case enablement
    • Use case building framework
  • Concepts
    • Core concepts
    • Low code development
    • Unified execution engine
    • Batch and stream processing
    • Continuous metrics
    • Key Joule data types
      • StreamEvent object
      • Contextual data
      • GeoNode
  • Tutorials
    • Getting started
    • Build your first use case
    • Stream sliding window quote analytics
    • Advanced tutorials
      • Custom missing value processor
      • Stateless Bollinger band analytics
      • IoT device control
  • FAQ
  • Glossary
  • Components
    • Pipelines
      • Use case anatomy
      • Data priming
        • Types of import
      • Processing unit
      • Group by
      • Emit computed events
      • Telemetry auditing
    • Processors
      • Common attributes
      • Filters
        • By type
        • By expression
        • Send on delta
        • Remove attributes
        • Drop all events
      • Enrichment
        • Key concepts
          • Anatomy of enrichment DSL
          • Banking example
        • Metrics
        • Dynamic contextual data
          • Caching architecture
        • Static contextual data
      • Transformation
        • Field Tokeniser
        • Obfuscation
          • Encryption
          • Masking
          • Bucketing
          • Redaction
      • Triggers
        • Change Data Capture
        • Business rules
      • Stream join
        • Inner stream joins
        • Outer stream joins
        • Join attributes & policy
      • Event tap
        • Anatomy of a Tap
        • SQL Queries
    • Analytics
      • Analytic tools
        • User defined analytics
          • Streaming analytics example
          • User defined analytics
          • User defined scripts
          • User defined functions
            • Average function library
        • Window analytics
          • Tumbling window
          • Sliding window
          • Aggregate functions
        • Analytic functions
          • Stateful
            • Exponential moving average
            • Rolling Sum
          • Stateless
            • Normalisation
              • Absolute max
              • Min max
              • Standardisation
              • Mean
              • Log
              • Z-Score
            • Scaling
              • Unit scale
              • Robust Scale
            • Statistics
              • Statistic summaries
              • Weighted moving average
              • Simple moving average
              • Count
            • General
              • Euclidean
        • Advanced analytics
          • Geospatial
            • Entity geo tracker
            • Geofence occupancy trigger
            • Geo search
            • IP address resolver
            • Reverse geocoding
            • Spatial Index
          • HyperLogLog
          • Distinct counter
      • ML inferencing
        • Feature engineering
          • Scripting
          • Scaling
          • Transform
        • Online predictive analytics
        • Model audit
        • Model management
      • Metrics engine
        • Create metrics
        • Apply metrics
        • Manage metrics
        • Priming metrics
    • Contextual data
      • Architecture
      • Configuration
      • MinIO S3
      • Apache Geode
    • Connectors
      • Sources
        • Kafka
          • Ingestion
        • RabbitMQ
          • Further RabbitMQ configurations
        • MQTT
          • Topic wildcards
          • Session management
          • Last Will and Testament
        • Rest endpoints
        • MinIO S3
        • File watcher
      • Sinks
        • Kafka
        • RabbitMQ
          • Further configurations
        • MQTT
          • Persistent messaging
          • Last Will and Testament
        • SQL databases
        • InfluxDB
        • MongoDB
        • Geode
        • WebSocket endpoint
        • MinIO S3
        • File transport
        • Slack
        • Email
      • Serialisers
        • Serialisation
          • Custom transform example
          • Formatters
        • Deserialisers
          • Custom parsing example
    • Observability
      • Enabling JMX for Joule
      • Meters
      • Metrics API
  • DEVELOPER GUIDES
    • Setting up developer environment
      • Environment setup
      • Build and deploy
      • Install Joule
        • Install Docker demo environment
        • Install with Docker
        • Install from source
        • Install Joule examples
    • Joulectl CLI
    • API Endpoints
      • Mangement API
        • Use case
        • Pipelines
        • Data connectors
        • Contextual data
      • Data access API
        • Query
        • Upload
        • WebSocket
      • SQL support
    • Builder SDK
      • Connector API
        • Sources
          • StreamEventParser API
        • Sinks
          • CustomTransformer API
      • Processor API
      • Analytics API
        • Create custom metrics
        • Define analytics
        • Windows API
        • SQL queries
      • Transformation API
        • Obfuscation API
        • FieldTokenizer API
      • File processing
      • Data types
        • StreamEvent
        • ReferenceDataObject
        • GeoNode
    • System configuration
      • System properties
  • Deployment strategies
    • Deployment Overview
    • Single Node
    • Cluster
    • GuardianDB
    • Packaging
      • Containers
      • Bare metal
  • Product updates
    • Public Roadmap
    • Release Notes
      • v1.2.0 Join Streams with stateful analytics
      • v1.1.0 Streaming analytics enhancements
      • v1.0.4 Predictive stream processing
      • v1.0.3 Contextual SQL based metrics
    • Change history
Powered by GitBook
On this page
  • Prerequisites
  • Install a Joule Docker image from DockerHub​
  • Running a Joule Docker image in a container​
  • Exposed Ports
  • Use case deployment
  • Building your own Joule Docker image​

Was this helpful?

  1. DEVELOPER GUIDES
  2. Setting up developer environment
  3. Install Joule

Install with Docker

Use prebuilt images for quick setup

PreviousInstall Docker demo environmentNextInstall from source

Last updated 6 months ago

Was this helpful?

Joule core, processor and connector plugins maintained by are available as images and distributed via .

Using a prebuilt Docker image to install Joule has a few benefits: it already includes core, all processors, data connectors and pinned versions of all their dependencies. By contrast, gradle build takes longer to run, and will always install the latest compatible versions of every dependency.

Prerequisites

  • You've installed Docker. For more information, see the site.

  • You understand which data connectors(s) you need. For more information, see Connector Catalog

  • You have a general understanding of the Joule, processing pipeline, developing locally in the command line interface (CLI).

Install a Joule Docker image from DockerHub

Official joule docker images are hosted on . We maintain images and tags for every version of the platform, as well as two tags that update as new versions as released:

  • latest: Latest overall version of Joule + connectors and processors

  • <Major>.<Minor>.latest: Latest patch of Joule + plugins for <Major>.<Minor> version family. For example, 1.0.latest includes the latest patches for Joule Platform v1.0

Install an image using the docker pull command:

docker pull fractalworks/joule-daemon:<version_tag>

Running a Joule Docker image in a container

The ENTRYPOINT for Joule Docker images is the command joule. You can bind-mount your project to /app/joule and use Joule as normal:

docker run -d --name=appName \
-p 1098:1098 \
-v $PWD/conf:/app/joule/conf \
-v $PWD/userlibs:/app/joule/userlibs \
-v $PWD/logs:/app/joule/logs: \
-v $PWD/data:/app/joule/data: \
--user joule \
--env-file $PWD/conf/app.env \
fractalworks/joule-daemon:latest

Notes:

Modify $PWD to the directory where the configuration, user jars, logs and data are located on the local machine. Joule container will use these directories to configure the process and to write log and data files.

  • Bind-mount sources must be an absolute path

  • You may need to make adjustments to the docker networking setting depending on the specifics of your data sources or hosts.

Exposed Ports

Two key ports are exposed to interact with the running Joule process.

  • 1098 JMX port Joule provides monitoring standard JMX beans, accessible on localhost port 1098. The following port is configured to enable remote monitoring through the use of tools such as JConsole, DataDog, VisualVm etc,.

Use case deployment

Dedicated Use case mode

Dedicated use cases are configured through the app.env. This sets up the required data sources, including reference data, use case processing and target publishing sinks.

conf directory

The app.env file defines the key configurations for Joule process to start. Each file specifies key components to load, register and startup.

Example:

SOURCEFILE=${PWD}/conf/sources/eventStream.yaml
ENGINEFILE=${PWD}/conf/usecases/eventAnonymisation.yaml
PUBLISHFILE=${PWD}/conf/publishers/anonymisedEventFile.yaml

Additionally, reference data sources, for event enrichment, can be added the following to app.env

REFERENCEDATA=${PWD}/conf/joules/sources/appReferenceData.yaml

Typically the yaml files will reside within the conf directory.

userlibs directory

Custom user components are deployed to the Joule classpath from the userlibs directory.

log directory

Joule generates standard log4j logging file, using logback binding, which is configured to exist beyond the lifetime of the container. A default logback.xml is provided.

data directory

Joule has the ability to generate event files which are persisted to a configured volume mount. This directory is not required if event data is not saved to file.

In particular, the Dockerfile supports building images:

  • Images that all plugins maintained by Fractalworks Labs

Please note that, if you go the route of building your own Docker images, we are unable to offer dedicated support for custom use cases. If you run into problems, you are welcome to open an issue in the Joule Core repository. If many users are requesting the same enhancement, we will tag the issue help_wanted and invite community contribution.

9080 Management RestAPI Use case deployment and management api, see api reference to learn more.

A Joule container can be started in one of two modes; either as a dedicated use case or as a flexible Joule use case agent. Below describes how to start using the dedicated use case mode, whereas for information on running as a see the getting started project .

Building your own Joule Docker image

If the pre-made images don't fit your use case, we also provide a and that can be used to build custom images in a variety of ways.

Fractalworks
Docker
GitLab packages
Docker
​
DockerHub
​
documentation
documentation
​
Dockerfile
README