Enrichment

Enrich events using connected contextual data, metrics and analytics

Overview

Enrichment is a common practice across industries and Joule makes it simpler.

In advanced streaming scenarios, it's often necessary to add extra contextual data to improve processing accuracy and decision-making.

By using low-latency data stores, streaming events can be quickly enriched with up-to-date information, ensuring fast and responsive data retrieval.

The key concepts and the anatomy of enrichment DSL will deep dive in the concept of enrichment.

Key conceptsAnatomy of enrichment DSL

Use cases

Here are some use cases of how it can be applied in stream processing:

1. Customer data

   Adding information such as data bundle plans or credit ratings.

2. Company metrics

   Enriching streams with market ratings or previous day stock sensitivities.

3. Technical operations

   Including data like cell tower signal coverage or machine utilisation limits.

4. Geofencing

   Enriching streams with static geofences for places of interest.

5. Dynamic triggers

   Incorporating real-time business triggers like SLAs or stock price levels.

6. Domain-specific metrics

   Adding general KPIs or metrics for domain-level insights.

Available enrichment options

Contextual data, usually stored in data marts or warehouses changes slowly. Integrating it into a high-throughput streaming platform can cause processing delays.

These processing delays are often due to the I/O overhead from the network and storage layers. To tackle these challenges, Joule offers a set of optimised solutions. Following enrichers are available.

Metrics

Enrich events with pre-calculated metrics to facilitate further pipeline processing

Dynamic contextual data

Use fast, connected data stores to quickly enrich in-motion events with up-to-date context

Static contextual data

Enrich events with essential slow changing data from Joule’s in-memory database.