Observability

Joule uses JMX and Metrics API to make processes observable and to enable monitoring

Overview

Joule offers robust observability features designed for monitoring each component in real-time.

This system-wide observability is achieved through JMX (Java Management Extensions) and a Metrics API, enabling users to track events, processing status and data movement across processors, connectors and storage.

By leveraging a standard monitoring pattern, JMX beans offer visibility into the state of all active processes.

Below is an example configuration for enabling JMX beans, illustrating how Joule’s system architecture exposes these metrics.

JMX beans UI

Key features of observability in Joule

1. JMX integration Enables remote monitoring and management of processes.

2. Metrics API Provides real-time counters for events processed, failed, discarded and more.

3. Automated Monitoring Each component (data pipelines, processor, connectors, storage) has default metrics enabled, facilitating quick access to performance insights.

4. Configurable Users can customize monitoring parameters for tailored observability.

Metrics

The following metrics are enabled by default and are accessible via JMX beans:

• Received Count of events received by the component.

• Processed Number of events successfully processed.

• Failed Events that failed during processing.

• Discarded Events discarded based on specified rules.

• Ignored Events ignored by the system.

• Average processing latency Measures the average time taken to process events.

Activate observability and monitoring

Enabling JMX for Joule

Activate JMX monitoring for Joule with configurable settings

Meters

Automated metrics monitor for Joule pipelines: data pipelines, processors, transport, storage

Metrics API

Tracks component events and data flow with counters

Additional information

• Learn about the JMX technology

• Learn about how to use the monitoring and management using JMX technology