# Send on delta

## Objective

Filter duplicate events from the processing stream to ensure that **only unique events** are processed once when specified attribute values change.

This filter uses an **efficient send-on-delta** algorithm, [bloom filter](https://www.geeksforgeeks.org/bloom-filters-introduction-and-python-implementation/), that looks at user-defined fields to determine the new received event has changed.

{% hint style="success" %}
Only **unique events** that show these changes are sent on to the next processing stage.
{% endhint %}

## Uses

There are various uses for this filter such as:

1. <mark style="color:green;">**In retail**</mark>\
   Configurable *inform on update* consumer pattern.\
   Track changes to inventory stock levels and trigger alerts only when there is an update in the quantity of a particular product.
2. <mark style="color:green;">**In finance**</mark>\
   Only perform stream processing on key field changes.\
   Monitor key financial metrics, such as stock prices or exchange rates, and generate alerts when a significant field update occurs.
3. <mark style="color:green;">**Healthcare**</mark>\
   Trigger alerting based upon a field update.\
   Observe critical patient metrics and notify relevant personnel only when there’s a change in specific key health indicators.&#x20;

### Proactive filter replacement

Overtime the likelihood of false positives occurring increases with the number of events processed. This is mitigated by triggering an **automatically filter update** when the False Positive Probability (`fpp`) value is breached.

Any `StreamEvent` with the `deltaBreachedFPP` field indicates a filter change after the event is published.

Furthermore the following attributes can be used to fine tune when to reset the filter:

* `reset by time delay`
* `reset by event count`

{% hint style="info" %}
Both of the reset functions can be applied with one being executed before the other.
{% endhint %}

## Examples & DSL attributes

{% tabs %}
{% tab title="Automatic replacement" %}
This configuration will replace the internal bloom filter once `fpp` value is breached.&#x20;

```yaml
send on delta:
  fields:
    - customerId
    - cash_balance
    - available_credit
  expected unique events: 100000
  fpp: 0.98
```

{% endtab %}

{% tab title="Proactive replacement" %}
This configuration will replace the internal bloom filter on one of three conditions:

* `fpp` value has breached 0.98.
* Time since filter creation is passed 15 minutes.
* Number of events seen has breached 50,000.

```yaml
send on delta:
  fields:
    - customerId
    - cash_balance
    - available_credit
  expected unique events: 100000
  fpp: 0.98
  reset by time delay: 900
  reset by event count: 500000
```

{% endtab %}
{% endtabs %}

### Attributes schema

<table><thead><tr><th width="193">Attribute</th><th width="217">Description</th><th width="219">Data Type</th><th data-type="checkbox">Required</th></tr></thead><tbody><tr><td>fields</td><td>Javascript expression that returns a boolean value</td><td>List&#x3C;String></td><td>true</td></tr><tr><td>expected unique events</td><td>It's important to calculate this accurately; otherwise, false positives will occur as the number nears or exceeds the set limit. A minimum of 1000 events is required to create the filter.</td><td>Long<br>Default: 1000</td><td>false</td></tr><tr><td>fpp</td><td>Desired false positive probability. must be positive and less than 1.0</td><td><p>Double</p><p>Default: 0.97</p></td><td>false</td></tr><tr><td>reset by time delay</td><td>Reset the bloom filter after a set number of seconds. </td><td><p>Long</p><p>Default: 0</p></td><td>false</td></tr><tr><td>reset by event count</td><td>Reset the bloom filter after a set number of events processed. </td><td><p>Integer</p><p>Default: 0</p></td><td>false</td></tr></tbody></table>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.fractalworks.io/joule/components/processors/filters/send-on-delta.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.