Caching architecture

Caching is a key enabler for low-latency contextual data access

Overview

The Joule architecture integrates a client cache into the process, allowing real-time data access from a distributed managed caching service.

This solution is generally only needed whenever a processes need to be triggered intra-day whenever key data changes at source. This is a common pattern with larger enterprise solution architectures and is also becoming more applicable to scale-ups.

Apache Geode reference solution

The reference architecture solution is built using Apache Geode although solutions such as Redis and Hazelcast are good candidates too and would be considered in the future.

Quick introduction to Apache Geode

Apache Geode is a recognised enterprise grade caching solution battle tested across multiple industries often in highly complex environments.

Over 20+ years the product has been deployed within healthcare, financial services, telco and many other industries and thus has many architecture deployment patterns.

Its key value proposition is providing a low latency and high throughput data to client services using a multi-region distributed cluster solution, along with vertical scaling techniques. Geode implement CAP theorem proving the user to decide upon which two CAP guarantees to follow.

The Joule implementation provides the eventing callbacks to bind the backing database to the distributed cache cluster using Geode's cache listener APIs. Any changes with the cluster are reflected with the embedded Joule client cache. And thus the client cache always has an updated image of the data stored locally.

Implementation

1. Set up the Apache Geode cluster, see Geode documentation.

2. Configure client cache connection, see Contextual data for more information on setup.

geode stores:
    name: mobile reference data
    connection:
        locator address: 192.168.86.39
        locator port: 41111
    stores:
      mobiledevices:
        region: mobile-devices
        keyClass : java.lang.String
        gii: true

1. Bind to the store within the enricher processor by using its logical mapping name. The enricher using assignment binds to the underlying store.

enricher:
  fields:      
    deviceManufacturer:
      by key: tac
      with values: [deviceManufacturer, year_released]
      using: deviceStore 
      
  stores:
    deviceStore:
      store name: mobiledevices