Enrichment
Enrich streaming events with up-to-date contextual data using low latency data stores
In sophisticated streaming scenarios, the execution of advanced use cases frequently necessitates supplemental contextual data for driving processing. The enrichment solution pattern is typically deployed by leveraging low-latency data stores to ensure swift and responsive data retrieval and enrichment.
Across industries enrichment is a standard process, below sets out example use cases within a stream processing context.
Example use cases
Customer data bundle plan, credit rating etc;
Company market rating, previous day stock sensitives etc;
Cell tower signal coverage, machine utilisation scaling limits etc;
Static Geofences for places of interest
Dynamic business trigger parameters (i.e. SLAs, stock price levels...)
General domain level static KPI / metrics
Available enrichment implementations
Reference data typically resides in a data mart or warehouse and changes slowly, integrating it into a high-throughput streaming platform can lead to processing latency challenges. This is often attributed to the I/O overhead induced by the network and storage layer. Joule has provided a set of solutions to address these concerns.
Metrics
Enrich events with pre-calculated metrics to facilitate further pipeline processing
In-motion Reference Data
Use linked low latency data stores to enrich events with fresh reference data
Last updated