Join attributes & policy

Join attributes to control and optimise performance

Stream join examples

Inner stream join

streams join:
  expression: "sitevisits.customerId == webpage.adclicks.customerId"
  merge events: true
  left policy:
    time to live: 30 minutes
  right policy:
    delete on join: true

Outer stream join

streams join:
  expression: "sitevisits.customerId *= webpage.adclicks.customerId"
  merge events: true
  left policy:
    time to live: 30 minutes
  right policy:
    delete on join: true

These attributes define how the join is performed with respect to the expression, how to create the emitted event structure etc.

Attributes schema

Attribute	Description	Data Type	Required
expression	Simple join expression that evaluates two streams	String
merge events	Flag that either merge event attributes into a flattened event structure or places each event	Boolean Default: true
event type	User defined type of emitted event	String Default: left type - right type (i.e. typeA-typeB)
left / right join	See Join attribute section	Policy attribute Default

Policy Attributes

Fine grained control can be applied to how joins are handled through the use of the policy attribute.

This becomes relevant for large and fast event stream with respect to memory and processing overhead.

Default configuration

1. delete on join is set to false and therefore will join events until the time to live setting is honoured.

2. time to live is set for 60 minutes per stored event. This is refreshed every time a new event is received with the same join attribute value.

Attributes schema

Attribute	Description	Data Type	Required
delete on join	Delete event from state management on a successful join	Boolean Default: false
time to live	Expire configuration for event stored within the state management system. Supported time units include nanoseconds, microseconds, milliseconds, seconds, hours and days	Formatted string Default: 60 minutes