# Architecture

## Overview

Joule’s data source interface allows processors to **access and utilise** contextual data seamlessly across the platform.

Upon startup, the Joule runtime **connects to each configured data source**, assigning each source a logical name and making it available to all processors.

This setup is managed through a **single configuration file**, which specifies the necessary contextual data stores ([see Configuration section](https://docs.fractalworks.io/joule/components/contextual-data/configuration)).

## Architecture

To maintain high performance in scenarios with **large event throughputs**, it is essential to choose the right data source implementation.

{% hint style="warning" %}
Poorly optimised data sources can hinder the **efficiency of the processing pipeline**, particularly during high-frequency data access
{% endhint %}

To **mitigate performance issues and reduce the load** from out-of-process I/O operations, Joule recommends caching contextual data locally within the process, especially in high-read environments.

This approach **optimises data retrieval speed and enhances the overall processing efficiency**, enabling smoother handling of real-time events.

### High level architecture

The Joule solution enables external tools to plugin to the Joule processing using the **Reference Data Interface**. 

<figure><img src="https://3062398388-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FUU6FZlV07ZD90OzbzGww%2Fuploads%2FDcOhyrLPNBOeDW8zqaoH%2FScreenshot%202023-09-18%20at%2016.07.42.png?alt=media&#x26;token=90e2bb4c-43c2-4a98-8242-dafb5edf326f" alt=""><figcaption><p>Streaming Prediction example</p></figcaption></figure>