Enrich streaming events with up-to-date contextual data using low latency data stores

In sophisticated streaming scenarios, the execution of advanced use cases frequently necessitates supplemental contextual data for driving processing. The enrichment solution pattern is typically deployed by leveraging low-latency data stores to ensure swift and responsive data retrieval and enrichment.

Across industries enrichment is a standard process, below sets out example use cases within a stream processing context.

Example use cases

  • Customer data bundle plan, credit rating etc;

  • Company market rating, previous day stock sensitives etc;

  • Cell tower signal coverage, machine utilisation scaling limits etc;

  • Static Geofences for places of interest

  • Dynamic business trigger parameters (i.e. SLAs, stock price levels...)

  • General domain level static KPI / metrics

Available enrichment implementations

Reference data typically resides in a data mart or warehouse and changes slowly, integrating it into a high-throughput streaming platform can lead to processing latency challenges. This is often attributed to the I/O overhead induced by the network and storage layer. Joule has provided a set of solutions to address these concerns.


Enrich events with pre-calculated metrics to facilitate further pipeline processing

In-motion Reference Data

Use linked low latency data stores to enrich events with fresh reference data

Static Reference Data

Enrich events with static reference data

Last updated