The objective of this project is to demonstrate Joule real-time analytics capabilities using time and event based window aggregate functions. Six use case files have been created that demonstrates various platform capabilities.
External market data events are generated using a basic simulator based upon a static nasdaq end of day market data file, published on to Kafka quotes topic. Events are consumed and processed by Joule with resulting analytics published into a InfluxDB time series bucket. Thereafter Grafana visualisations or and other suitable tool can be applied.

Joule features demonstrated

  1. 1.
    Sliding windows (i.e. time and event based)
  2. 2.
    Group by aggregate functions
  3. 3.
    User defined analytics using SDK (Bollinger Bands)
  4. 4.
    Event filtering
  5. 5.
    Event projection
  6. 6.
    InfluxDB transport
  7. 7.
    Kafka publisher and consumer transports
  8. 8.
    Kafka event transformers
  9. 9.
    SQLTap that captures processed events within stream
  10. 10.
    RestAPI to access raw and processed events
Good to know: depending on the product you're building, it can be useful to explicitly document use cases. Got a product that can be used by a bunch of people in different ways? Maybe consider splitting it out!

Setting up

Clone the project repository is hosted on Gitlab
By building the example locally you will have the flexibility to create new use cases, analytics, processors, transports and contribute back the to project. Build the project in the root directory by following the below instructions.
First build the project
gradle clean build && cd build/distributions
&& unzip
&& cd fractalworks-banking-example-1.0
&& chmod +x bin/*.sh
Start joule processor
Events will be generated, published, consumed, process and finally published to InfluxDB and ready to be visualised in Grafana.

Stock Quote Simulator

Simulated stock quote events are created using the StockQuoteSimulator driver. A total of 7514 stock quotes are generated and published on each processing cycle. Update the StockQuoteSimulator class for additional fields and processing requirements. The conf/csv/nasdaq.csv file used to prime the simulator.

Quote Event

The following fields are available on each quote event
- ingestTime
- eventTime
- symbol
- bid
- ask
- volume
- volatility
- date
The Kafka configuration can be found here conf/simulator/kafkapublisher.yaml.

Use case

The use case is defined using the Joule low-code approach.
name: standardQuoteAnalyticsStream
enabled: true
validFrom: 2020-01-01
validTo: 2025-12-31
eventTimeType: EVENT_TIME
sources: [ nasdaq_quotes_stream ]
- timeWindow:
emittingEventType: coreQuoteAnalytics
MIN: [ ask ]
MAX: [ bid ]
SUM: [ volume ]
MEAN: [ volatility ]
type: slidingTime
slideSize: 500
windowSize: 2500
expression: "symbol, ask_MIN, bid_MAX, volume_SUM, volatility_MEAN"
- symbol