Architecture
Understand how contextual data integrates within Joule
Last updated
Understand how contextual data integrates within Joule
Last updated
Fractalworks has many years developing real-time event based solutions and products that leverage this technology. Please contact us for for details on how we can help you.
Joule’s data source interface allows processors to access and utilise contextual data seamlessly across the platform.
Upon startup, the Joule runtime connects to each configured data source, assigning each source a logical name and making it available to all processors.
This setup is managed through a single configuration file, which specifies the necessary contextual data stores (see Configuration section).
To maintain high performance in scenarios with large event throughputs, it is essential to choose the right data source implementation.
Poorly optimised data sources can hinder the efficiency of the processing pipeline, particularly during high-frequency data access
To mitigate performance issues and reduce the load from out-of-process I/O operations, Joule recommends caching contextual data locally within the process, especially in high-read environments.
This approach optimises data retrieval speed and enhances the overall processing efficiency, enabling smoother handling of real-time events.
The Apache Geode architecture within Joule leverages a connected client cache that initially links through the Geode locator, then connects directly to cluster members. This approach supports on-demand loading of dynamic contextual data, enhancing data retrieval speeds and ensuring scalability for high-throughput environments.
Given its importance, it’s recommended to document the architecture separately from the configuration file for clearer guidance and to simplify maintenance.
