# User defined scripts

## Objective

The analytics processor feature set is extended to host external scripts within the Joule processing context. Enabling existing scripting assets to be leveraged within a streaming context.

Apply external analytical scripts within a real-time streaming context.

{% hint style="info" %}
Currently Javascript ECMAScript 2024, graalpy, Python 3 and 3.11 are supported
{% endhint %}

<figure><img src="/files/gUicjU2h8RLYmvf2EsdV" alt=""><figcaption><p>Graphic data flow of a functioning UDF</p></figcaption></figure>

## Execution models

This guide explores different execution models for integrating scripts and functions within a stream processing environment.

We will discuss how to use pre-existing JavaScript or Python scripts, custom functions, and expressions to perform complex analytics and calculations on event data.

1. [<mark style="color:green;">**Script and expression**</mark>](#script-and-expression)\
   Learn how to incorporate existing JavaScript files and use expressions that reference event attributes, ideal when you want to calculate new variables on-the-fly.
2. [<mark style="color:green;">**Script and function**</mark>](#script-and-function)\
   Understand how to define custom functions that process entire events, allowing for deeper calculations by accessing multiple event attributes.

### Script and Expression

Use this option when you have pre-existing scripts which you want to leverage within a stream processing context.

```yaml
scripting:
  script: ./scripts/js/myCustomFunctions.js
  expression: bid - squareRoot(`${ask}` / `${bid}`)
  assign to: new_event_variable
```

#### **User defined function**

This example applies the `bid` and `ask` event attributes to the expression.

Note, the current implementation requires the script to be provided using the `js` extension due to the way the expression is defined.

All other variables are provided as global within the execution context.

{% tabs %}
{% tab title="Javascript" %}

```javascript
export function squareRoot(num) {
    if (num < 0) return NaN;
    return Math.sqrt(num);
}
```

{% endtab %}

{% tab title="Python" %}

```python
import math

def squareRoot(num):
    if num < 0:
        return float('nan')
    return math.sqrt(num)
```

{% endtab %}
{% endtabs %}

### Script and function

This option builds upon the previous example, but uses a method which is supplied with a `StreamEvent` which can further call imported scripts.

All other variables are provided as global within the execution context.

```yaml
scripting:
  script: ./scripts/js/usedDefinedCustomFunctions.js
  language: javascript
  function: totalCompensationPackage
  assign to: total_compensation
```

#### **User defined function**

The event is automatically passed as a method parameter per execution call.

This method uses the Javascript modules which are defined using the `mjs` file extension.&#x20;

{% tabs %}
{% tab title="Javascript" %}

```javascript
export function totalCompensationPackage(event) {
    return event.getValue("salary") +
           event.getValue("bonus") + 
           event.getValue("benefits");
}
```

{% endtab %}

{% tab title="Python" %}

```python
def totalCompensationPackage(event):
    return (event.get("salary", 0) +
            event.get("bonus", 0) +
            event.get("benefits", 0))
```

{% endtab %}
{% endtabs %}

## Attributes schema

The same attributes described in User defined analytics are applicable, see [documentation](/joule/components/analytics/analytic-tools/user-defined-analytics/user-defined-analytics.md).

<table><thead><tr><th width="144">Attribute</th><th width="296.87890625">Description</th><th width="174.76953125">Data Type</th><th data-type="checkbox">Required</th></tr></thead><tbody><tr><td>language</td><td>Language runtime to use to execute required execution definitions. Currently Javascript (javascript or js) and Python (python or py) are supported </td><td>String<br>Default: js</td><td>false</td></tr><tr><td>expression</td><td>Math expression without an assignment variable. Required if method has not been provided.</td><td>String</td><td>false</td></tr><tr><td>script</td><td>Path of the script to load within the Joule processing context</td><td>String</td><td>false</td></tr><tr><td>function</td><td>Function to execute within the provided script using a StreamEvent as a parameter</td><td>String</td><td>false</td></tr><tr><td>expand all</td><td>Add all StreamEvent attributes to the processing context as independ variables</td><td>Boolean<br>Default: true<br></td><td>false</td></tr><tr><td>options</td><td>Provide language specific GraalVm engine options. See <a href="#graalvm-options">GraalVM Options</a> section</td><td>Map&#x3C;String, String><br>Default: Language specific </td><td>false</td></tr></tbody></table>

### GraalVM Options

To change the behaviour of the GraalVm engine language specific options can be passed in using DSl syntax

{% tabs %}
{% tab title="Javascript" %}
**Example**

```yaml
scripting:
  script: ./scripts/js/usedDefinedCustomFunctions.js
  language: javascript
  function: totalCompensationPackage
  assign to: total_compensation
  options:
    js.strict: true
    js.nashorn-compat: true
    js.ecmascript-version: 2024
    js.esm-eval-returns-exports: true
```

\
**Default options**

If no options are provided the following options are applied. For a complete list of available options review the GraalVM Js [documentation](https://www.graalvm.org/latest/reference-manual/js/Options/).

| Key                         | Value |
| --------------------------- | ----- |
| js.esm-eval-returns-exports | true  |
| js.nashorn-compat           | true  |
| js.ecmascript-version       | 2024  |
| {% endtab %}                |       |

{% tab title="Python" %}
**Example**

```yaml
scripting: 
  script: ./scripts/python/usedDefinedCustomFunctions.py
  language: python
  function: totalCompensationPackage
  assign to: total_compensation
  options:
    python.PythonPath: ./org.graalvm.python.vfs/venv
    python.Executable: ./org.graalvm.python.vfs/venv/bin/
    python.ForceImportSite: true
```

**Default options**

If no options are provided the following options are applied.&#x20;

| Key                    | Value |
| ---------------------- | ----- |
| python.ForceImportSite | true  |

**Note**

Please contact Fractalworks to help you leverage this feature. \
\
These resources will need to be understand to fully leverage this capability:

* GraalPy [documentation](https://www.graalvm.org/python/)
* Package [compatibility](https://www.graalvm.org/python/compatibility/?version=v241) <br>
  {% endtab %}
  {% endtabs %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.fractalworks.io/joule/components/analytics/analytic-tools/user-defined-analytics/user-defined-scripts.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.