# Architecture

## Overview

Joule’s data source interface allows processors to **access and utilise** contextual data seamlessly across the platform.

Upon startup, the Joule runtime **connects to each configured data source**, assigning each source a logical name and making it available to all processors.

This setup is managed through a **single configuration file**, which specifies the necessary contextual data stores ([see Configuration section](/joule/components/contextual-data/configuration.md)).

## Architecture

To maintain high performance in scenarios with **large event throughputs**, it is essential to choose the right data source implementation.

{% hint style="warning" %}
Poorly optimised data sources can hinder the **efficiency of the processing pipeline**, particularly during high-frequency data access
{% endhint %}

To **mitigate performance issues and reduce the load** from out-of-process I/O operations, Joule recommends caching contextual data locally within the process, especially in high-read environments.

This approach **optimises data retrieval speed and enhances the overall processing efficiency**, enabling smoother handling of real-time events.

### High level architecture

The Joule solution enables external tools to plugin to the Joule processing using the **Reference Data Interface**. 

<figure><img src="/files/oxvgEWgEadcEbu4dyrXJ" alt=""><figcaption><p>Streaming Prediction example</p></figcaption></figure>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.fractalworks.io/joule/components/contextual-data/architecture.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.