Comment on page

Environment installation

Installation
The following describes how to set up a local environment mainly using docker images. I have provided docker-compose files to start the environment processes.
Start up infrastructure
Startup Postgres, Confluent Kafka, InfluxDB and Grafana using the provided scripts located under their respective infra directories
docker-compose -f infra/postgres/docker-compose.yml up -d
docker-compose -f infra/confluent/docker-compose.yml up -d
docker-compose -f infra/influx-grafana/docker-compose.yml up -d
Postrges configurations
To use Postgres you will need to perform three simple tasks, define a target schema, copy jdbc postgres jar and link the databaseStandardAnalytics.yaml file in the bin/app.env file.
cp infra/postgres/postgresql-42.5.0.jar lib/
Example database schema file
CREATE TABLE IF NOT EXISTS quote_analytics (
    id              SERIAL PRIMARY KEY,
    symbol          VARCHAR(10),
    ask_EMA         float8,
    bid_EMA         float8
    );
Example database publisher file
publisher:
  name: standardAnalyticsDatabasePublisher
  source: standardQuoteAnalyticsStream
  sinks:
    - sqlPublisher:
        jdbcDriver: org.postgresql.Driver
        host: jdbc:postgresql://localhost:5432
        database: jouleTestDB
        table: quote_analytics
        includeTimestamps: false
        properties:
          ssl: false
          user: joule
          password: joulepwd
Influx and Grafana configurations
The infra/influx-grafana/.env file contains all the required configuration setting for InfluxDB and Grafana. Processed events will be stored using the below org and bucket.
user: influx
password: influxdb
organisation: banking
bucket: nasdaqBaseAnalytics
In the event of changing the setting these need to be replicated in the influxDB publishing conf/publishers/influxdbStandardAnalytics.yaml file.
Start Quote generator
Once the infrastructure is running it's time to start running the use case.
Start quote generator
./bin/startQuotePublisher.sh

Use Cases - Previous

Banking

Tumbling window analytics

Last modified 10mo ago