System properties

Configure and enable core Joule features based on deployment needs

Overview

Joule internal setting can be custom configured to suit the needs of your environment.

This guide allows precise control over Joule’s setup for optimised deployment.

Key sections include:

1. General configuration Set metadata and enable core services like the internal database and REST API.

2. Service enabling Control features such as event lineage, internal database, and REST API.

3. Database settings Configure Joule’s embedded DuckDB for metrics and state queries, including memory limits and thread usage.

4. Rest API Customise REST API options, including file handling, security, and SSL settings.

Example configuration

# General Rest API details 
joule.version=1.1.0
joule.contact.name=Lyndon Adams
joule.contact.email=[email protected]
joule.contact.url=https://fractalworks.io

# Enable service switches
joule.lineage.enable=false
joule.internaldb.enable=true
joule.restapi.enable=true

# Internal database setting
joule.db.path=./db/joule.db
joule.db.memoryOnly=false
joule.db.memory_limit='512MB'
joule.db.access_mode=READ_WRITE
joule.db.threads=2
joule.db.worker_threads=2
joule.db.default_order='DESC'

# RestAPI setting
joule.rest.port=9080

# Directory to write files that exceed the in-memory limit
joule.rest.downloadPath=downloads

# The maximum individual file size allowed In GB
joule.rest.maxFileSize=2

# The maximum file size to handle in memory in MB
joule.rest.InMemoryFileSize=100

# The maximum size of the entire multipart request in GB
joule.rest.totalRequestSize=5

# SSL config
joule.rest.enableSSL=false
joule.rest.pemCertPath=./conf/cert.pem
joule.rest.keyCertPath=./conf/key.pem

Joule CLI switch

Apply the below command line switch to the Joule process when using a custom joule.properties file. Setting within this file will override the default setting.

To override the default setting provide a single configuration file and use the -c command line switch to specify the file path.

-c /app/joule/config/joule.properties

Enable services

To enable services using default setting the following

# Enable service switches
joule.lineage.enable=false
joule.internaldb.enable=true
joule.restapi.enable=true

Event lineage

Any event processor that changes the state of the passed event can have this action recorded within the event. There is a small memory overhead to keep changed state.

joule.lineage.enable=false

This is currently an experimental feature which is undergoing further development to determine its usefulness within a stream processing context. Therefore it is set to disabled as a default setting.

Internal database

Joule ships with an embedded database, DuckDB, that provides core features such as metrics processing, event tapping and external state queries. This feature is enabled by default.

joule.internaldb.enable=true

See internal database configuration for further details.

Rest API

Joule ships with an embedded Rest API that provides external query and Web socket APIs. See Rest API documentation for further details. This feature is enabled by default.

joule.restapi.enable=true

See Rest API configuration for further details.

Internal database configuration

Joule ships with an in-memory database to support event capturing, metrics and ad-hoc queries for processors.

Example

# Internal database setting
joule.db.path=./db/joule.db
joule.db.memoryOnly=false
joule.db.memory_limit=4GB
joule.db.access_mode=READ_WRITE
joule.db.threads=8
joule.db.default_order=DESC

Attributes schema

Property	Description	Default
joule.db.path	Relative path to the Joule process include database file name.	default: ./db/joule.db
joule.db.memoryOnly	Boolean flag set to true an in-memory database is created. Note that for an in-memory database no data is persisted to disk.	default: false
joule.db.memory_limit	The maximum memory of the database system (e.g. 1GB)	default: 512MB
joule.db.access_mode	Access mode of the database (AUTOMATIC, READ_ONLY or READ_WRITE).	default: READ_WRITE
joule.db.threads	The number of total worker threads used by the database.	default: 2
joule.db.default_order	The query set order type used when none is specified (ASC or DESC).	default: DESC

Rest API Configuration

Joule enables Rest API when the in-memory database is enabled. Dynamic endpoints are created to support data exports and paginated queries. See Rest API documentation for further information.

# RestAPI setting
joule.rest.port=9080

# Directory to write files that exceed the in-memory limit
joule.rest.downloadPath=downloads

# The maximum individual file size allowed In GB
joule.rest.maxFileSize=2

# The maximum file size to handle in memory in MB
joule.rest.InMemoryFileSize=100

# The maximum size of the entire multipart request in GB
joule.rest.totalRequestSize=5

# SSL config
joule.rest.enableSSL=false
joule.rest.pemCertPath=./conf/cert.pem
joule.rest.keyCertPath=./conf/key.pem

Attributes schema

Property	Description	Default
joule.rest.port	Rest API port	default: 9080
joule.rest.downloadPath	Directory the system rest consumer will download files to ready for data ingestion See Rest API documentation	default: downloads
joule.rest.maxFileSize	The maximum individual file size allowed In GB	default: 2GB
joule.rest.InMemoryFileSize	The maximum file size to handle in memory in MB	default: 100MB
joule.rest.totalRequestSize	The maximum size of the entire multipart request in GB	default: 5GB
joule.rest.enableSSL	If you want to secure your Joule process with SSL set this flag to true. Supported certificate formats are PKCS#12 (.p12) and PKCS#8 (.pem) and Java KeyStore (.jks).	default: false
joule.rest.pemCertPath	Certificate from a trusted CA or a self-signed certificate.	default: /app/joule/certs/cert.pem
joule.rest.keyCertPath	Private key	default: /app/joule/certs/key.pem