Sliding window analytics

Sliding windows are useful to gain analytic insights over a period whereby some of the events contribute to the insight, these windows are either time-based or a fixed count of events.

---

This example provides a demonstration of the available OOTB aggregate functions Joule provides that work with sliding windows.

Use case configuration

File: app-slidingWindowAnalytics.env

SOURCEFILE=conf/sources/stockQuoteStream.yaml
ENGINEFILE=conf/usecases/slidingWindowAnalytics.yaml
PUBLISHFILE=/conf/publishers/fileStandardAnalytics.yaml

Pipeline configuration

This pipeline will compute a sliding window function on a single event type based on symbol CVCO

stream:
  name: standardQuoteAnalyticsStream
  enabled: true
  validFrom: 2020-01-01
  validTo: 2025-12-31
  eventTimeType: EVENT_TIME
  sources: [ nasdaq_quotes_stream ]

  processing unit:
    pipeline:
      - time window:
          emitting type: slidingQuoteAnalytics
          aggregate functions:
            FIRST: [ask]
            LAST: [ ask ]
            MIN: [ ask ]
            MAX: [ bid ]
            SUM: [ volume ]
            MEAN: [ volatility ]
            HARMONIC_MEAN: [ volatility ]
            VARIANCE: [ volatility ]
            STDEV: [ volatility ]
          policy:
            type: slidingTime
            slide: 500
            window size: 2500

  select:
    expression: "symbol, ask_FIRST, ask_LAST, ask_MIN, bid_MAX, volume_SUM, volatility_MEAN, volatility_HARMONIC_MEAN, volatility_VARIANCE, volatility_STDEV"

  group by:
    - symbol