Getting started

All you need to get started with Joule.

The tutorial is built using Joule v1.2.1-SNAPSHOT

Overview

What this tutorial is about

This tutorial aims to teach how to run a real-time banking example with your first Joule.

Who this tutorial is intended for

We have tailored this tutorial for anyone who wants to learn how to use Joule for the first time and who wishes to learn how to deploy a first use case given that your environment is setup already.

What we will learn

By the end of this tutorial, we will have learned:

1. How to start and stop Joule.

2. Deploy a first real-time use case.

3. Access the Redpanda interface to see the deployed use case in action.

Prerequisites

1. Understanding of how to navigate with the command line through directories. Not sure how to navigate with command line? Follow this tutorial.

2. Joule requires Graalvm to be installed, see environment setup..

3. Postman is installed. Don’t have Postman? Follow this tutorial.

4. Docker or any other alternative to run containers.

5. Git is installed. Don’t have Git? Follow this tutorial.

Follow this tutorial to setup your environment

Optional

Only install the below if you do not want to run Docker images of vendor software:

1. Redpanda is installed. Don’t have Redpanda? Follow this tutorial.

Implementation

Step 1 - Start up our environment

The following steps will guide us on setting up our environment to deploy our images and containers with Joule.

1. We will clone the repository with this command.

git clone https://gitlab.com/joule-platform/fractalworks-stream-gettingstarted.git

git clone [email protected]:joule-platform/fractalworks-stream-gettingstarted.git

1. Once cloned go in to the directory.

cd fractalworks-stream-gettingstarted

1. Now we need to setup the environment by build and copying required files in to the quickstart directory by executing gradle buildJouleDemoEnv, see below.

(base) ~/projects/fractalworks-stream-gettingstarted git:[dev]
gradle buildJouleDemoEnv

And should generate the below output

## This command should generate these messages
Configuration on demand is an incubating feature.

> Task :fractalworks-stream-gettingstarted:iot-demo:setupIoTDemo
IoT Demo copying directories to quickstart
- bin
- data
- conf
- simulator
- libs --> userlibs

> Task :fractalworks-stream-gettingstarted:banking-demo:setupBankingDemo
Banking Demo copying directories to quickstart
- bin
- data
- conf
- libs --> userlibs

> Task :fractalworks-stream-gettingstarted:telco-demo:setupTelcoDemo
Telco Demo copying directories to quickstart
- libs --> userlibs

1. We will navigate to the quickstart directory.

# Change in to directory
cd quickstart

1. We will make sure that Docker or an alternative is running.

We are now ready to start running Joule. To start Joule, we need to run the following command which will start building our images and containers.

Within the getting started root directory four projects are provided. In the case of this use case, we will run the following command from the ~/quickstart directory:

# Startup Joule
(base) ~/projects/fractalworks-stream-gettingstarted/quickstart git:[dev]

./startJoule

The command will start up the following containers ready for use case deployment:

• Joule Daemon

• Joule Database

• Redpanda

We should expect this output

Expected result when images built

Expected result when containers built

Step 2 - Run the Getting Started example

The banking demo directories provides a set of examples in the form of a postman collection and environment. These can be found under the examples directory.

We will now get started running the banking Getting Started demo example using Postman.

1. First we will import the use case demos and environment files from the banking-demo examples directory. Import the PostMan collection by selecting “Import” from the Collections tab and select the file Joule.postman_environment.json from fractalworks-stream-gettingstarted/banking-demo/examples

Postman collection

1. We will set the Postman environment to Joule.

Setting the environment

1. From the Getting started \ Deploy folder click Run folder from the menu.

Deploying the example

1. Finally execute the run order by clicking the Run Joule - Banking demo button.

Running the example

This will deploy the source, stream, sink and the use case binding definition to the platform.

Note on a restart these setting will be rehydrated and will automatically start.

We should expect this output

Expected result in Postman

Step 3 - Will start the quote publisher

The banking demo is provided with a quote simulator which is seeded using

./bin/banking/startQuotePublisher

This command will generate a continuous set of quote and publish them to the quotes topic. We will see how to access the topic on the next step.

We should expect this output

Press return to continue.

Expected result when the quote publisher has started

Step 4 - Now review our use case outputs

Once the processing pipeline completes its tasks, the resulting events are published on to the analytics_view topic ready for further consumption (i.e. distribution, dashboard, dynamic portfolio risk measures etc).

To learn how to navigate the interface, look at this tutorial.

Access the Redpanda interface

We should expect this output

Expected result when Redpanda interface loaded

Step 5 - Stop the quotes simulator

Once we are ready to shut down Joule, we will first stop the quotes publisher by executing the command below.

./bin/stopQuotePublisher

We can now undeploy the use case and its linked components by executing the Run Joule - Banking demo under the Undeploy use case folder.

In case we want to execute complete shut down of Joule, we can instead follow the steps below.

We should expect this output

Expected result when the quote publisher has been stopped

Step 6 - Shut the environment down

Once quotes publisher has been stopped the Joule environment can now be shutdown. We have 2 options to shut Joule down.

Complete shutdown and clean environment

We will execute the following command to shut down the environment.

./shutdownAndCleanJoule

We should expect this output

Expected result shutdown and clean Joule

Only shutdown

What did we learn?

We covered essential steps to run our first Joule including cloning the repository and running a first streaming banking use case via Postman.

In summary we have learned:

1. How to publish continuous data streams with the quote simulator.

2. How to see our use case in execution in the Redpanda interface.

3. How to review and process analytics outputs through our default use case.

4. How to stop, shut down the Joule environment and clean up the system.

What’s next?

Now that we have learned the basics about Joule, it is time to learn how to create our own use case.