# Dynamic contextual data

{% hint style="info" %}
Joule provides an [Apache Geode](https://geode.apache.org/) connector with an embedded client cache.
{% endhint %}

## Objective

Enriching event data with the latest contextual information is crucial for obtaining **real-time business insights**.

Since this data often resides in data marts or warehouses and changes slowly, **integrating** it into a high-speed streaming platform can **lead to delays**, primarily due to the pressure on network and storage systems. Joule **addresses this challenge** by incorporating an **embedded cache** within its architecture.

Upon startup, developers can specify the necessary datasets and **customise the cache's functionality**. This is an enterprise-level solution which supports advanced use cases, **enabling real-time insights** and rapid decision-making.

Two types of enrichment strategies:

1. <mark style="color:green;">**Key based enrichment**</mark>\
   Use a event attribute as a key to retrieve the value object from the key-value store.
2. <mark style="color:green;">**Query based enrichment**</mark>\
   A flexible approach to retrieve the target value using an object query.

## Uses

There are various uses for this filter such as:

1. <mark style="color:green;">**In retail**</mark>\
   integrate real-time sales data with inventory levels using key-based enrichment to ensure that stock levels are updated instantly, allowing for accurate inventory tracking and efficient restocking processes.
2. <mark style="color:green;">**In healthcare**</mark>\
   Utilise query-based enrichment to enrich patient event data with the latest contextual information, such as medication history and allergies, ensuring healthcare providers have comprehensive insights for timely decision-making.
3. <mark style="color:green;">**In e-commerce**</mark>\
   Utilise key-based enrichment to enhance user interactions with contextual product recommendations based on browsing history and past purchases, optimising the shopping experience and increasing conversion rates.

## Example & DSL attributes

The enricher allows users to enhance events with multiple data elements through advanced mapping.

The code defines an enricher that enhances events with data from two stores `mobiledevices` and `mobilecontracts`:

1. <mark style="color:green;">**deviceManufacturer**</mark>\
   Retrieves `deviceManufacturer` and `year_released` using `tac` as the key from the `mobiledevices` store.
2. <mark style="color:green;">**modelDetails**</mark>\
   Retrieves all details about the model as an object using `tac` as the key from the `mobiledevices` store.
3. <mark style="color:green;">**contractedDataBundle**</mark>\
   Uses an SQL query to fetch data from `dataBundleStore` based on the `imsi` field, retrieving all attributes.

```yaml
enricher:
  fields:      
    deviceManufacturer:
      by key: tac
      with values: [deviceManufacturer, year_released]
      using: deviceStore

    modelDetails:
      by key: tac
      as object: true   
      using: deviceStore

    contractedDataBundle:
      by query:  "select * from /userBundle where imsi = ?"
      query fields: [imsi]
      all attributes: true
      using: dataBundleStore

  stores:
    deviceStore:
      store name: mobiledevices

    dataBundleStore:
      store name: mobilecontracts
```

## Key based enrichment <a href="#key-based-enrichment" id="key-based-enrichment"></a>

Using the key-based lookup approach requires defining a target key-value store. Joule provides an [Apache Geode](https://geode.apache.org/) connector with an embedded client cache.

**Return contextual data as a linked map of key-value pairs**

```yaml
deviceManufacturer:
    by key: tac
    with values: [deviceManufacturer, year_released]
    using: deviceStore
```

#### Provide return contextual data as a linked object

```yaml
modelDetails:
  by key: tac
  as object: true   
  using: deviceStore
```

### Attributes schema

<table><thead><tr><th width="168">Attribute</th><th width="392">Description</th><th width="174">Data Type</th></tr></thead><tbody><tr><td>by key</td><td>Uses the value within the passed event as a lookup key on the linked store</td><td>String</td></tr></tbody></table>

## Query based enrichment

Similarly, the query-based approach requires defining a target key-value store.

```yaml
contractedDataBundle:
    by query:  "select * from /userBundle where imsi = ?"
    query fields: [imsi]
    all attributes: true
    using: dataBundleStore
```

### Attributes schema

<table><thead><tr><th width="168">Attribute</th><th width="392">Description</th><th width="174">Data Type</th></tr></thead><tbody><tr><td>by query</td><td>Uses the provided query to enrich the event</td><td>String</td></tr></tbody></table>

### OQL Note

Joule utilises the Apache Geode platform for **dynamic data solutions**, offering an **enterprise-grade approach** that is well-understood and thoroughly tested.

Geode employs a query syntax based on Object Query Language (OQL). While OQL shares many similarities with SQL, it has key differences; for instance, OQL does not support all SQL features like aggregates but allows

* Querying complex object graphs
* Object attributes
* Invoking object methods