Apache Geode

High-performance caching platform for static and slow-moving contextual data

Fractalworks has extensive experience developing real-time event based solutions and products that applies this technology.

Please contact us for for details on how we can help you.

Overview

Apache Geode integrates with Joule as a high-performance, in-memory key-value store. As an essential part of the the platform, it provides low-latency access to contextual data for advanced stream processing.

Known for its reliability and scalability, Geode has supported large-scale enterprise solutions for over 20 years.

In Joule, Geode enables immediate updates from a distributed cluster, allowing processors to use the latest data in real-time without manual refreshes. To learn more about the features of Apache Geode, we invite you to read this article for a really good overview.

Driver details: org.apache.geode:geode-core:1.15.1

Storage regions can be primed on startup using either a custom query or a getAll function otherwise data is fetched on cache misses.

Furthermore, for cached data elements, data updates are immediately propagated from the connected distributed cluster to the Joule process thereby enabling up-to-date contextual data.

Use cases

Geode is ideal for handling various types of static or slow-moving reference data, critical in scenarios requiring low-latency access and high concurrency. Common examples include:

Typical use cases include:

  1. Geographic and regional data Postal codes, country codes (e.g., ISO-366).

  2. Device and market data Mobile manufacture models, market exchange codes.

  3. Currency data Currency codes for international transactions.

  4. Machine learning models Pre-trained models for real-time predictions.

  5. Static variables and codes Charge codes, car VINs, pre-computed data for analysis.

  6. Pattern matching Regex patterns for validating inputs, such as phone numbers.

Architecture

Joule leverages a connected Geode client cache that initially links through the Geode locator, and then after connects directly to cluster members. This approach supports on-demand loading of dynamic contextual data, enhancing data retrieval speeds and ensuring scalability for high-throughput environments.

It’s recommended to review Apache Geode documentation to understand the distributed data cluster architecture. This will inform you of the benefits of the solution.

Example & DSL attributes

The following example is a configuration for integrating Apache Geode as a data source within Joule:

reference data:
 ...
    - geode stores:
          name: us markets
          connection:
              locator address: 192.168.86.39
              locator port: 41111
          stores:
            nasdaqIndex:
              region: nasdaq-companies
              keyClass : java.lang.String
              gii: true
            holidays:
              region: us-holidays
              keyClass : java.lang.Integer

Explanation

This configuration specifies Geode as a data source, linking two key regions: nasdaq-companies and us-holidays.

The gii (get-initial-image) option for nasdaqIndex ensures data is primed on startup, minimising cache misses. By connecting via the provided locator address and port, Joule gains access to the Geode distributed cluster, ensuring the most up-to-date data in the processing pipeline.

Attributes schema

AttributeDescriptionData TypeRequired

name

Logical name of reference data set for given target connection

String

locator address

Locator process IP address

String Default: localhost

locator port

The port the cluster locator listens for cluster and client connections

Integer Default: 41111

stores

Map of logical names to region configurations. See Region attributes

List of logical storage name to Geode storage region

PreviousMinIO S3NextConnectors

Last updated