# User defined scripts

## Objective

The analytics processor feature set is extended to host external scripts within the Joule processing context. Enabling existing scripting assets to be leveraged within a streaming context.

Apply external analytical scripts within a real-time streaming context.

{% hint style="info" %}
Currently Javascript ECMAScript 2024, graalpy, Python 3 and 3.11 are supported
{% endhint %}

<figure><img src="https://3062398388-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FUU6FZlV07ZD90OzbzGww%2Fuploads%2FOAbUwxc60UBiLu29VrG9%2Fimage.png?alt=media&#x26;token=1e68f872-3bf5-455d-ae31-dbd5b6facecd" alt=""><figcaption><p>Graphic data flow of a functioning UDF</p></figcaption></figure>

## Execution models

This guide explores different execution models for integrating scripts and functions within a stream processing environment.

We will discuss how to use pre-existing JavaScript or Python scripts, custom functions, and expressions to perform complex analytics and calculations on event data.

1. [<mark style="color:green;">**Script and expression**</mark>](#script-and-expression)\
   Learn how to incorporate existing JavaScript files and use expressions that reference event attributes, ideal when you want to calculate new variables on-the-fly.
2. [<mark style="color:green;">**Script and function**</mark>](#script-and-function)\
   Understand how to define custom functions that process entire events, allowing for deeper calculations by accessing multiple event attributes.

### Script and Expression

Use this option when you have pre-existing scripts which you want to leverage within a stream processing context.

```yaml
scripting:
  script: ./scripts/js/myCustomFunctions.js
  expression: bid - squareRoot(`${ask}` / `${bid}`)
  assign to: new_event_variable
```

#### **User defined function**

This example applies the `bid` and `ask` event attributes to the expression.

Note, the current implementation requires the script to be provided using the `js` extension due to the way the expression is defined.

All other variables are provided as global within the execution context.

{% tabs %}
{% tab title="Javascript" %}

```javascript
export function squareRoot(num) {
    if (num < 0) return NaN;
    return Math.sqrt(num);
}
```

{% endtab %}

{% tab title="Python" %}

```python
import math

def squareRoot(num):
    if num < 0:
        return float('nan')
    return math.sqrt(num)
```

{% endtab %}
{% endtabs %}

### Script and function

This option builds upon the previous example, but uses a method which is supplied with a `StreamEvent` which can further call imported scripts.

All other variables are provided as global within the execution context.

```yaml
scripting:
  script: ./scripts/js/usedDefinedCustomFunctions.js
  language: javascript
  function: totalCompensationPackage
  assign to: total_compensation
```

#### **User defined function**

The event is automatically passed as a method parameter per execution call.

This method uses the Javascript modules which are defined using the `mjs` file extension.&#x20;

{% tabs %}
{% tab title="Javascript" %}

```javascript
export function totalCompensationPackage(event) {
    return event.getValue("salary") +
           event.getValue("bonus") + 
           event.getValue("benefits");
}
```

{% endtab %}

{% tab title="Python" %}

```python
def totalCompensationPackage(event):
    return (event.get("salary", 0) +
            event.get("bonus", 0) +
            event.get("benefits", 0))
```

{% endtab %}
{% endtabs %}

## Attributes schema

The same attributes described in User defined analytics are applicable, see [documentation](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/user-defined-analytics/user-defined-analytics).

<table><thead><tr><th width="144">Attribute</th><th width="296.87890625">Description</th><th width="174.76953125">Data Type</th><th data-type="checkbox">Required</th></tr></thead><tbody><tr><td>language</td><td>Language runtime to use to execute required execution definitions. Currently Javascript (javascript or js) and Python (python or py) are supported </td><td>String<br>Default: js</td><td>false</td></tr><tr><td>expression</td><td>Math expression without an assignment variable. Required if method has not been provided.</td><td>String</td><td>false</td></tr><tr><td>script</td><td>Path of the script to load within the Joule processing context</td><td>String</td><td>false</td></tr><tr><td>function</td><td>Function to execute within the provided script using a StreamEvent as a parameter</td><td>String</td><td>false</td></tr><tr><td>expand all</td><td>Add all StreamEvent attributes to the processing context as independ variables</td><td>Boolean<br>Default: true<br></td><td>false</td></tr><tr><td>options</td><td>Provide language specific GraalVm engine options. See <a href="#graalvm-options">GraalVM Options</a> section</td><td>Map&#x3C;String, String><br>Default: Language specific </td><td>false</td></tr></tbody></table>

### GraalVM Options

To change the behaviour of the GraalVm engine language specific options can be passed in using DSl syntax

{% tabs %}
{% tab title="Javascript" %}
**Example**

```yaml
scripting:
  script: ./scripts/js/usedDefinedCustomFunctions.js
  language: javascript
  function: totalCompensationPackage
  assign to: total_compensation
  options:
    js.strict: true
    js.nashorn-compat: true
    js.ecmascript-version: 2024
    js.esm-eval-returns-exports: true
```

\
**Default options**

If no options are provided the following options are applied. For a complete list of available options review the GraalVM Js [documentation](https://www.graalvm.org/latest/reference-manual/js/Options/).

| Key                         | Value |
| --------------------------- | ----- |
| js.esm-eval-returns-exports | true  |
| js.nashorn-compat           | true  |
| js.ecmascript-version       | 2024  |
| {% endtab %}                |       |

{% tab title="Python" %}
**Example**

```yaml
scripting: 
  script: ./scripts/python/usedDefinedCustomFunctions.py
  language: python
  function: totalCompensationPackage
  assign to: total_compensation
  options:
    python.PythonPath: ./org.graalvm.python.vfs/venv
    python.Executable: ./org.graalvm.python.vfs/venv/bin/
    python.ForceImportSite: true
```

**Default options**

If no options are provided the following options are applied.&#x20;

| Key                    | Value |
| ---------------------- | ----- |
| python.ForceImportSite | true  |

**Note**

Please contact Fractalworks to help you leverage this feature. \
\
These resources will need to be understand to fully leverage this capability:

* GraalPy [documentation](https://www.graalvm.org/python/)
* Package [compatibility](https://www.graalvm.org/python/compatibility/?version=v241) <br>
  {% endtab %}
  {% endtabs %}