Apache Geode
High-performance caching platform for static and slow-moving contextual data
Last updated
Was this helpful?
High-performance caching platform for static and slow-moving contextual data
Last updated
Was this helpful?
Apache Geode integrates with Joule as a high-performance, in-memory key-value store. As an essential part of the the platform, it provides low-latency access to contextual data for advanced stream processing.
Known for its reliability and scalability, Geode has supported large-scale enterprise solutions for over 20 years.
In Joule, Geode enables immediate updates from a distributed cluster, allowing processors to use the latest data in real-time without manual refreshes. To learn more about the features of Apache Geode, we invite you to read this for a really good overview.
Storage regions can be primed on startup using either a custom query or a getAll
function otherwise data is fetched on cache misses.
Furthermore, for cached data elements, data updates are immediately propagated from the connected distributed cluster to the Joule process thereby enabling up-to-date contextual data.
Geode is ideal for handling various types of static or slow-moving reference data, critical in scenarios requiring low-latency access and high concurrency. Common examples include:
Typical use cases include:
Geographic and regional data Postal codes, country codes (e.g., ISO-366).
Device and market data Mobile manufacture models, market exchange codes.
Currency data Currency codes for international transactions.
Machine learning models Pre-trained models for real-time predictions.
Static variables and codes Charge codes, car VINs, pre-computed data for analysis.
Pattern matching Regex patterns for validating inputs, such as phone numbers.
Joule leverages a connected Geode client cache that initially links through the Geode locator, and then after connects directly to cluster members. This approach supports on-demand loading of dynamic contextual data, enhancing data retrieval speeds and ensuring scalability for high-throughput environments.
The following example is a configuration for integrating Apache Geode as a data source within Joule:
This configuration specifies Geode as a data source, linking two key regions: nasdaq-companies
and us-holidays
.
The gii
(get-initial-image) option for nasdaqIndex
ensures data is primed on startup, minimising cache misses. By connecting via the provided locator address and port, Joule gains access to the Geode distributed cluster, ensuring the most up-to-date data in the processing pipeline.
name
Logical name of reference data set for given target connection
String
locator address
Locator process IP address
String Default: localhost
locator port
The port the cluster locator listens for cluster and client connections
Integer Default: 41111
stores
Map of logical names to region configurations. See Region attributes
List of logical storage name to Geode storage region
It’s recommended to review to understand the distributed data cluster architecture. This will inform you of the benefits of the solution.