Emit computed events

Select and filter output events for publishing

Overview

Publish event query projections to downstream systems, with options to select, refine and calculate event attributes and filter the final projected event.

The following function can be applied to the projection expression

1. Event attribute selection Select event attributes to be included in the final results.

2. Add metric data Apply calculated metrics within the projection.

3. Apply simple math functions Use simple math function on the selected attributes.

4. Filter resulting projected data Use the having filter to narrow the result set, similar to the SQL having clause.

5. Attribute renaming using an alias Rename attributes to ease system integrations.

Using the select expression

To bring this to life, we shall use the following select query and final stream event that will be queried.

Query

stream:
  ...
  emit:
    eventType: buy_alert
    select:
        "
        venue.name 'venue',
        symbol, 
        ask,
        bid,
        ask - bid 'spread',
        details.*,
        BidMovingAverage.avg_ask|avg_bid;where symbol = {symbol}
        "
    having: "symbol =='IBM' && spread > 0.05"

Source event

The following event example will be used to demonstrate the select query and having filter.

# Source processed event
streamEvent:
  ...
  attributes:
      symbol: IBM
      ask: 101
      bid: 100
      details: {
         vol: 50
         liq: 60   
      }
      venue: {
         name: nasdaq
      }

Output event

This will produce the following result which is published to the connected consumers.

# Result object
event type: buy_alert
  venue: nasdaq
  symbol: IBM
  ask: 101
  bid: 100
  spread: 1
  vol: 50
  liq: 60
  avg_ask: 101.34
  avg_bid: 100.12

Using metrics

A key feature is the ability to use and embed calculated metrics within the select expression.

This enables the following advanced capabilities:

• Ability to add pre-calculated metrics within the select projection.

• Perform calculations using event attributes and metrics.

• Embed metrics values to be applied to within the having clause logic

For further information on how this feature is used go to the apply metrics documentation.

Example

stream:
  ...
  emit:
    ...
    select:
        "
        ...
        # Adding metrics attributes
        BidMovingAverage.avg_ask|avg_bid;where symbol = {symbol}
        "
    having:
       "symbol =='IBM' && avg_ask > 101.15"

Attributes schema

Format	Result	Description
BidMovingAverage.avg_ask;where symbol = {symbol}	101.34	Query the BidMovingAverage table where the event symbol joins the matched latest row for attribute avg_ask
BidMovingAverage.avg_ask|avg_bid;where symbol = {symbol}	101.34, 100.12	Query the BidMovingAverage table where the event symbol joins the matched latest row for attributes avg_ask and avg_bid
BidMovingAverage.avg_ask;where symbol = {symbol} AND venue=${venue.name}	101.24, 100.22	Query the BidMovingAverage table where the event symbol and venue joins the matched latest row for the attribute avg_ask

having clause

This is a Javascript expression based filter that evaluates the expression against the resulting select projection result.

If the expression condition is met the resulting event is published otherwise it is dropped.

This is an optional property

# Only publish IBM events with an avg_ask greater than 101.15
stream:
  ...
  emit:
  ...
    having: "symbol == 'IBM' && avg_ask > 101.15"

retain

This boolean attribute informs the system to cache and conflate the computed select projection event which can be used across processor processing contexts.

Each select statement field is prefixed with the emiited_ tag and stored within the defined group by context object (i.e. symbol ). This context is cached ready for next processing state either on a successful having clause or simply without.

Example

This example uses the having clause to demonstrate how to use the previous emitted event to deterime if the evednt should be filtered.

stream:
  processing unit:
    pipeline:
    ...
     - analytic:
          expression: "(bid_MAX - ask_MIN) / bid_MAX"
          assign to: bid_spread
          
  emit:
    select: symbol, ask_MIN, bid_MAX, volume_SUM, volatility_MEAN, bid_spread
    having: bid_spread > 0.015 && (bid_spread > (emitted_bid_spread || bid_spread))
    retain: true
  
  group by
     - symbol

Attributes schema

Attribute	Description	Data Type	Required
event type	Allows renaming of the event type in the published output. The default appends the final event type attribute to "_view"	String Default: <eventType>_view
select	SQL-style command that lets users specify which fields to include. Users can perform calculations (i.e., averages) on fields as well and it supports integration with the metrics engine for advanced calculations. Users can add metrics to the event output by specifying metric expressions in the select clause	String
having	This is a Javascript expression filter that evaluates the expression against the resulting select project. If the expression passes the resulting event is presented to publishing sinks	String
retain	Store the select projection so that it can be used in next processing cycle.	Boolean Default: false