Key concepts
Enriching events with the latest contextual data is crucial for processing advanced real-time business insights
Last updated
Enriching events with the latest contextual data is crucial for processing advanced real-time business insights
Last updated
This article introduces high-level concepts about the enrichment processor, providing a foundational understanding of key topics and how they are used.
We will learn how Joule enables the linkage of contextual data to stream events within a processing pipelines that subscribe to source data and publish results.
This article covers:
Enrichment architecture An overview of enrichment architecture components and how they work together.
Supported data stores List of supported enrichment data stores.
Worked example Learn how to apply enrichment within a stream processing pipeline.
Out-of-the-box Joule provides a low-latency event enrichment solution for advanced use cases.
This core feature enables the user to focus on composing data requirements using a pragmatic enrichment query DSL.
The Joule contextual data architecture supports various low-latency in-memory data stores that reduce the inherent I/O overhead of out-of-the-box process databases and thus enable stream based enrichment.
Note contextual data refers is a generic term to refer to any associative data or calculated data point that can provide additional context to an event.
The architecture components work together to provide the necessary functions for a high-performance enrichment process.
User enrichment DSL Enables the user to define a field level enrichment query, response type and the data source using a flexible syntax.
Contextual data interface Provides the standardised access abstraction over the underlying data sources
In-memory data stores Access to the internal high-performance in-memory data store and also a reference implementation of a proven enterprise data cluster solution.
S3 data access Standardised S3 data access for cross cloud collaboration
These are the supported out-of-the-box enrichment stores Joule provides:
Joule DB
S3
Distributed caching cluster
