> For the complete documentation index, see [llms.txt](https://docs.fractalworks.io/joule/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.fractalworks.io/joule/components/contextual-data/architecture.md).

# Architecture

## Overview

Joule’s data source interface allows processors to **access and utilise** contextual data seamlessly across the platform.

Upon startup, the Joule runtime **connects to each configured data source**, assigning each source a logical name and making it available to all processors.

This setup is managed through a **single configuration file**, which specifies the necessary contextual data stores ([see Configuration section](/joule/components/contextual-data/configuration.md)).

## Architecture

To maintain high performance in scenarios with **large event throughputs**, it is essential to choose the right data source implementation.

{% hint style="warning" %}
Poorly optimised data sources can hinder the **efficiency of the processing pipeline**, particularly during high-frequency data access
{% endhint %}

To **mitigate performance issues and reduce the load** from out-of-process I/O operations, Joule recommends caching contextual data locally within the process, especially in high-read environments.

This approach **optimises data retrieval speed and enhances the overall processing efficiency**, enabling smoother handling of real-time events.

### High level architecture

The Joule solution enables external tools to plugin to the Joule processing using the **Reference Data Interface**. 

<figure><img src="/files/oxvgEWgEadcEbu4dyrXJ" alt=""><figcaption><p>Streaming Prediction example</p></figcaption></figure>