Joule
Search
K
Comment on page

Event Windows

Joule provides the ability to perform time or number of events window based analytics. A suite of standard window functions are provided out-of-the-box. Developers can create custom analytic functions which are deployable as discrete components on the classpath.
Tumbling
Sliding
Tumbling windows are a non-overlapping events arrays ordered by time. In a tumbling window, events are grouped in to a single window based on time or count. An event belongs to only one window. Events within a window are only processed once when a new window is presented for processing.
Sliding windows, unlike Tumbling windows, output events only for points in time when the content of the window actually changes. In other words, when an event enters or exits the window. So, every window has at least one event. Note events can belong to more than one sliding window.

Example

processing unit:
pipeline:
- time window:
emitting type: tumblingQuoteAnalytics
aggregate functions:
FIRST: [ask]
LAST: [ ask ]
MIN: [ ask ]
MAX: [ bid ]
SUM: [ volume ]
MEAN: [ volatility ]
VARIANCE: [ volatility ]
STDEV: [ volatility ]
policy:
type: tumblingTime
window size: 5000

Core Attributes

Attribute
Description
Data Type
Required
emitting type
name of the event
String
preprocessor
Window preprocessor
window listeners
Custom window listeners which are executed on window generation. See window listener documentation below
See documentation below.
aggregate functions
Standard set of aggragation functions based over a window using group by semantics
See documentation below.
policy
Type of window processing. Supported types:
  • Tumbling
    • tumblingTime
    • tumblingCount
  • Sliding
    • slidingTime
    • slidingCount
String
e.g. slidingCount

Preprocessor

Before a window is process the user is able to inject a preprocessing function. The function returns a new window which is subsequently processed by defined window listeners and aggregate functions.
import com.fractalworks.streams.core.data.streams.OrderedStreamEventWindow;
import com.fractalworks.streams.core.data.streams.StreamEvent;
import com.fractalworks.streams.core.data.streams.StreamTimeType;
import com.fractalworks.streams.core.data.streams.Window;
import com.fractalworks.streams.sdk.analytics.WindowPreprocessor;
/**
* Custom preprocessor
*/
public class MyCustomPreprocessor extends WindowPreprocessor {
public MyCustomPreprocessor(StreamTimeType streamTimeType, int count, int groupByKey) {
super(streamTimeType, count, groupByKey);
}
public Window compute(Window window) {
StreamEvent[] orderedEvents = // Do something interesting
return new OrderedStreamEventWindow(orderedEvents, groupbyKey, streamTimeType);
}
}

WindowListener API

Custom window listeners can be developed and deployed in to the Joule environment. See building and deploying custom components documentation for further information.

API

package com.fractalworks.streams.core.listeners;
import com.fractalworks.streams.core.data.Tuple;
import com.fractalworks.streams.core.data.streams.Context;
import com.fractalworks.streams.core.data.streams.Window;
import java.util.Map;
public interface WindowListener {
Tuple<String, Map<String, Object>> apply(Window window, Context context);
}

Aggregation Functions

Joule provides a set of standard aggregate functions that can be applied to a set of events within a triggered window.

Available functions

When the function applied the field the result is added in the returned stream events as an additional field e.g <field_name>_SUM
Type
Description
SUM
Sum of field values
MIN
Min of field value
MAX
Max of field value
MEAN
Mean of field value
VARIANCE
Variance of field value
STDEV
Standard deviation of field value
FIRST
First field value in window
LAST
Last field value in window
HARMONIC_MEAN
Harmonic mean of field value
GEOMETRIC_MEAN
Geometric mean of field value
PVARIANCE
Population variance of field value
SECOND_MOMENT
Seond moment of field value
SUM_SQRTS
Sum of square root of field value
SUM_LOGS
Sum of log of field value