# Send on delta

## Objective

Filter duplicate events from the processing stream to ensure that **only unique events** are processed once when specified attribute values change.

This filter uses an **efficient send-on-delta** algorithm, [bloom filter](https://www.geeksforgeeks.org/bloom-filters-introduction-and-python-implementation/), that looks at user-defined fields to determine the new received event has changed.

{% hint style="success" %}
Only **unique events** that show these changes are sent on to the next processing stage.
{% endhint %}

## Uses

There are various uses for this filter such as:

1. <mark style="color:green;">**In retail**</mark>\
   Configurable *inform on update* consumer pattern.\
   Track changes to inventory stock levels and trigger alerts only when there is an update in the quantity of a particular product.
2. <mark style="color:green;">**In finance**</mark>\
   Only perform stream processing on key field changes.\
   Monitor key financial metrics, such as stock prices or exchange rates, and generate alerts when a significant field update occurs.
3. <mark style="color:green;">**Healthcare**</mark>\
   Trigger alerting based upon a field update.\
   Observe critical patient metrics and notify relevant personnel only when there’s a change in specific key health indicators.&#x20;

### Proactive filter replacement

Overtime the likelihood of false positives occurring increases with the number of events processed. This is mitigated by triggering an **automatically filter update** when the False Positive Probability (`fpp`) value is breached.

Any `StreamEvent` with the `deltaBreachedFPP` field indicates a filter change after the event is published.

Furthermore the following attributes can be used to fine tune when to reset the filter:

* `reset by time delay`
* `reset by event count`

{% hint style="info" %}
Both of the reset functions can be applied with one being executed before the other.
{% endhint %}

## Examples & DSL attributes

{% tabs %}
{% tab title="Automatic replacement" %}
This configuration will replace the internal bloom filter once `fpp` value is breached.&#x20;

```yaml
send on delta:
  fields:
    - customerId
    - cash_balance
    - available_credit
  expected unique events: 100000
  fpp: 0.98
```

{% endtab %}

{% tab title="Proactive replacement" %}
This configuration will replace the internal bloom filter on one of three conditions:

* `fpp` value has breached 0.98.
* Time since filter creation is passed 15 minutes.
* Number of events seen has breached 50,000.

```yaml
send on delta:
  fields:
    - customerId
    - cash_balance
    - available_credit
  expected unique events: 100000
  fpp: 0.98
  reset by time delay: 900
  reset by event count: 500000
```

{% endtab %}
{% endtabs %}

### Attributes schema

<table><thead><tr><th width="193">Attribute</th><th width="217">Description</th><th width="219">Data Type</th><th data-type="checkbox">Required</th></tr></thead><tbody><tr><td>fields</td><td>Javascript expression that returns a boolean value</td><td>List&#x3C;String></td><td>true</td></tr><tr><td>expected unique events</td><td>It's important to calculate this accurately; otherwise, false positives will occur as the number nears or exceeds the set limit. A minimum of 1000 events is required to create the filter.</td><td>Long<br>Default: 1000</td><td>false</td></tr><tr><td>fpp</td><td>Desired false positive probability. must be positive and less than 1.0</td><td><p>Double</p><p>Default: 0.97</p></td><td>false</td></tr><tr><td>reset by time delay</td><td>Reset the bloom filter after a set number of seconds. </td><td><p>Long</p><p>Default: 0</p></td><td>false</td></tr><tr><td>reset by event count</td><td>Reset the bloom filter after a set number of events processed. </td><td><p>Integer</p><p>Default: 0</p></td><td>false</td></tr></tbody></table>