Install with Docker

Use prebuilt images for quick setup

Joule core, processor and connector plugins maintained by Fractalworks are available as Docker images and distributed via GitLab packages.

Using a prebuilt Docker image to install Joule has a few benefits: it already includes core, all processors, data connectors and pinned versions of all their dependencies. By contrast, gradle build takes longer to run, and will always install the latest compatible versions of every dependency.

Prerequisites

• You've installed Docker. For more information, see the Docker site.

• You understand which data connectors(s) you need. For more information, see Connector Catalog

• You have a general understanding of the Joule, processing pipeline, developing locally in the command line interface (CLI).

Install a Joule Docker image from DockerHub​

Official joule docker images are hosted on DockerHub. We maintain images and tags for every version of the platform, as well as two tags that update as new versions as released:

• latest: Latest overall version of Joule + connectors and processors

• <Major>.<Minor>.latest: Latest patch of Joule + plugins for <Major>.<Minor> version family. For example, 1.0.latest includes the latest patches for Joule Platform v1.0

Install an image using the docker pull command:

docker pull fractalworks/joule-daemon:<version_tag>

Running a Joule Docker image in a container​

The ENTRYPOINT for Joule Docker images is the command joule. You can bind-mount your project to /app/joule and use Joule as normal:

docker run -d --name=appName \
-p 1098:1098 \
-v $PWD/conf:/app/joule/conf \
-v $PWD/userlibs:/app/joule/userlibs \
-v $PWD/logs:/app/joule/logs: \
-v $PWD/data:/app/joule/data: \
--user joule \
--env-file $PWD/conf/app.env \
fractalworks/joule-daemon:latest

Notes:

Modify $PWD to the directory where the configuration, user jars, logs and data are located on the local machine. Joule container will use these directories to configure the process and to write log and data files.

• Bind-mount sources must be an absolute path

• You may need to make adjustments to the docker networking setting depending on the specifics of your data sources or hosts.

Exposed Ports

Two key ports are exposed to interact with the running Joule process.

• 9080 Management RestAPI Use case deployment and management api, see api reference documentation to learn more.

• 1098 JMX port Joule provides monitoring standard JMX beans, accessible on localhost port 1098. The following port is configured to enable remote monitoring through the use of tools such as JConsole, DataDog, VisualVm etc,.

Use case deployment

A Joule container can be started in one of two modes; either as a dedicated use case or as a flexible Joule use case agent. Below describes how to start using the dedicated use case mode, whereas for information on running as a see the getting started project documentation.

Dedicated Use case mode

Dedicated use cases are configured through the app.env. This sets up the required data sources, including reference data, use case processing and target publishing sinks.

conf directory

The app.env file defines the key configurations for Joule process to start. Each file specifies key components to load, register and startup.

Example:

SOURCEFILE=${PWD}/conf/sources/eventStream.yaml
ENGINEFILE=${PWD}/conf/usecases/eventAnonymisation.yaml
PUBLISHFILE=${PWD}/conf/publishers/anonymisedEventFile.yaml

Additionally, reference data sources, for event enrichment, can be added the following to app.env

REFERENCEDATA=${PWD}/conf/joules/sources/appReferenceData.yaml

Typically the yaml files will reside within the conf directory.

userlibs directory

Custom user components are deployed to the Joule classpath from the userlibs directory.

log directory

Joule generates standard log4j logging file, using logback binding, which is configured to exist beyond the lifetime of the container. A default logback.xml is provided.

data directory

Joule has the ability to generate event files which are persisted to a configured volume mount. This directory is not required if event data is not saved to file.

Building your own Joule Docker image​

If the pre-made images don't fit your use case, we also provide a Dockerfile and README that can be used to build custom images in a variety of ways.

In particular, the Dockerfile supports building images:

• Images that all plugins maintained by Fractalworks Labs

Please note that, if you go the route of building your own Docker images, we are unable to offer dedicated support for custom use cases. If you run into problems, you are welcome to open an issue in the Joule Core repository. If many users are requesting the same enhancement, we will tag the issue help_wanted and invite community contribution.