Install with Docker

Joule core, processor and connector plugins maintained by Fractalworks are available as a Docker images and distributed via GitLab packages.

Using a prebuilt Docker image to install Joule has a few benefits: it already includes core, all processors, data connectors and pinned versions of all their dependencies. By contrast, gradle build takes longer to run, and will always install the latest compatible versions of every dependency.

Prerequisites

• You've installed Docker. For more information, see the Docker site.

• You understand which data connectors(s) you need. For more information, see Connector Catalog

• You have a general understanding of the Joule, processing pipeline, developing locally in the command line interface (CLI). For more information, see Joule Overview.

---

Install a Joule Docker image from DockerHub​

Offical joule docker images is hosted on DockerHub. We maintain images and tags for every version of the platform, as well as two tags that update as new versions as released:

• latest: Latest overall version of Joule + connectors and processors

• <Major>.<Minor>.latest: Latest patch of Joule + plugins for <Major>.<Minor> version family. For example, 1.0.latest includes the latest patches for Joule Platform v1.0

Install an image using the docker pull command:

docker pull fractalworks/joule:<version_tag>

Running a Joule Docker image in a container​

The ENTRYPOINT for Joule Docker images is the command joule. You can bind-mount your project to /app/joule and use Joule as normal:

docker run -d --name=appName \
-p 1098:1098 \
-v $PWD/conf:/app/joule/conf \
-v $PWD/userlibs:/app/joule/userlibs \
-v $PWD/logs:/app/joule/logs: \
-v $PWD/data:/app/joule/data: \
--user joule \
--env-file $PWD/conf/app.env \
fractalworks/joule:latest

Notes:

Modify $PWD to the directory where the configuration, user jars, logs and data are located on the local machine. Joule container will use these directories to configure the process and to write log and data files.

• Bind-mount sources must be an absolute path

• You may need to make adjustments to the docker networking setting depending on the specifics of your data sources or hosts.

Exposed Ports

Joule provides monitoring standard JMX beans, accessible on localhost port 1098. The following port is configured to enable remote monitoring through the use of tools such as JConsole, DataDog, VisualVm etc,.

• 1098 JMX port

The JMX port will be automatically exposed when using docker run -P.

Configuration

A Joule container is configured through the app.env which sets up the required data sources, including reference data, use case processing and target publishing sinks.

conf directory

The app.env file defines the key configurations for Joule process to start. Each file specifies key components to load, register and startup.

Example:

SOURCEFILE=${PWD}/conf/sources/eventStream.yaml
ENGINEFILE=${PWD}/conf/usecases/eventAnonymisation.yaml
PUBLISHFILE=${PWD}/conf/publishers/anonymisedEventFile.yaml

Additionally, reference data sources, for event enrichment, can be added the following to app.env

REFERENCEDATA=${PWD}/conf/joules/sources/appReferenceData.yaml

Typically the yaml files will reside within the conf directory.

userlibs directory

Custom user components are deployed to the Joule classpath from the userlibs directory.

log directory

Joule generates standard log4j logging file, using logback binding, which is configured to exist beyond the lifetime of the container. A default logback.xml is provided.

data directory

Joule has the ability to generate event files which are persisted to a configured volume mount. This directory is not required if event data is not saved to file.

Building your own Joule Docker image​

If the pre-made images don't fit your use case, we also provide a Dockerfile and README that can be used to build custom images in a variety of ways.

In particular, the Dockerfile supports building images:

• Images that all plugins maintained by Fractalworks Labs

Please note that, if you go the route of building your own Docker images, we are unable to offer dedicated support for custom use cases. If you run into problems, you are welcome to open an issue in the Joule Core repository. If many users are requesting the same enhancement, we will tag the issue help_wanted and invite community contribution.