# Quickstart ***
{% hint style="success" %} Download the getting started project [here](https://gitlab.com/joule-platform/fractalworks-stream-gettingstarted/-/tree/dev?ref_type=heads). Clone the repository and follow the instructions to run the examples {% endhint %} When you start working with Joule you will be editing files locally using a code editor and running use case examples using [Postman](https://www.postman.com/). If you prefer to build your projects within an Integrated Development Environment (IDE) clone the existing banking example project from [here](https://gitlab.com/joule-platform/fractalworks-banking-examples). ## Prerequisites Getting started with Joule has minimal requirements to getting started but to take full advantage of the platform some key technical capabilities would be needed. ### Step 1 Clone the [getting started](https://gitlab.com/joule-platform/fractalworks-stream-gettingstarted/-/tree/dev?ref_type=heads) project to a local directory by using the following command: ```bash git clone https://gitlab.com/joule-platform/fractalworks-stream-gettingstarted.git ``` ### Step 2 Install the required platform tools if you want to experiment with Joule SDK, see the [setting up the environment](/joule/developer-guides/setting-up-developer-environment/environment-setup.md) document. **Note:** To configure and run Joule it is important that you know some basics of the Terminal. In particular, you should understand general bash commands to navigate through the directory structure of your computer easily. ## Running an example Joule has provided the necessary scripts and configurations on getting a use case running using either a Docker image or a local unpacked installation, we shall use the local installation to get you familiar with the general directory structure, this will benefit your understanding of the provided Docker image. ### 1. Start up the environment For this you need to change in to the `quickstart` directory and run a single command. ```bash cd quickstart ./startupJoule.sh ``` This will start up the following containers ready for use case deployment: * Joule Daemon * Joule Database * [Redpanda](https://www.redpanda.com/) (Lightweight Kafka implementation) ### 2. Deploy a use case Both the banking and telco demo directories provide a set of examples in the form of a postman collection and environment. These can be found under the `examples` directory. Now lets get you started running the banking `Getting Started` demo example using Postman. 1. First import the use case demos and environment files from the banking-demo `examples` directory 2. Set the environment to Joule. 3. From the `Getting started \ Deploy` folder click `Run folder` from the menu. 4. Finally execute the `run order` by clicking the `Run Joule - Banking demo` button This will deploy the source, stream, sink and the use case binding definition to the platform. Note on a restart these setting will be rehydrated and will automatically start. ### 3. Start the quote simulator This will generate simulated quotes based upon the provided nasdaq csv info file. ```bash quickstart % ./bin/startQuotePublisher.sh Starting Quotes Simulator v1.2.1-SNAPSHOT ProcessId 52506 appending output to nohup.out ``` ### 4. View the results ```bash docker exec -it joule-gs-redpanda-0 rpk topic consume analytics_view --num 1 { "topic": "analytics_view", "key": "\u0000\u00015L", "value": "{\"symbol\":\"PHD\",\"time\":1706625828336,\"askFirst\":15.420006518870125,\"askLast\":15.420006518870125}", "timestamp": 1706625828093, "partition": 0, "offset": 0 } ``` ### Or from the Redpanda UI Use this [link](http://localhost:8080/topics/analytics_view?p=-1\&s=5\&o=-1#messages) to access the console

Redpanda UI

### 5. Stopping the processes ```bash quickstart % ./bin/stopQuotePublisher.sh Quote simulator closed quickstart % shutdownJoule.sh Joule processed closed ``` Note there are many other examples within the `getting started` project. These are described within the README.md files. ### Optional Step - Setup the environment If you choose to build the environment for development purposes this is done by simply running the below command, please ensure you have the correct build environment set out in the [setting up the environment](/joule/developer-guides/setting-up-developer-environment/environment-setup.md) document. ```bash gradle buildJouleDemoEnv ``` This will build both the banking and telco example components, and copy across the configurations, libraries, and setup the demo environment. ## Whats going on? The banking getting started use case demonstrates core features that are reusable across all use cases; connecting to data, processing and distributing events. Hence, we shall start with a simple use case that demonstrates how to get yourself running with the platform using the following repeatable steps. The use case subscribes to a Kafka quotes topic, get the high and low price per symbol using tumbling windows and then publish resulting event onto the analytics\_view topic. ## 1. Connect and subscribe to a streaming data source Define either one or more event sources using provided [data source connectors](/joule/components/connectors/sources.md). #### Overview of the definition * `nasdaq_quotes_event_stream` is the logical name for the source definition * Subscribe to events using the `quotes` Kafka topic * Received events are deserialised using a user defined parser in to a Joule `StreamEvent` to enable processing ### Example Kafka subscription {% code fullWidth="false" %} ```yaml kafkaConsumer: name: nasdaq_quotes_stream cluster address: joule-gs-redpanda-0:9092 consumerGroupId: nasdaq topics: - quotes deserializer: parser: com.fractalworks.examples.banking.data.QuoteToStreamEventParser key deserializer: org.apache.kafka.common.serialization.IntegerDeserializer value deserializer: com.fractalworks.streams.transport.kafka.serializers.object.ObjectDeserializer properties: partition.assignment.strategy: org.apache.kafka.clients.consumer.StickyAssignor max.poll.records: '7000' fetch.max.bytes: '10485760' ``` {% endcode %} ## 2. Process events Use case processing is defined as a pipeline of processing stages. Joule provides a set of OOTB processors, see [documentation](broken://pages/id1UsoobNKuH4pP06AHz), along with a [SDK](/joule/developer-guides/builder-sdk/processor-api.md) to enable developers to extend the platform capabilities. #### Overview of the definition * `basic_tumbling_window_pipeline` is used as the logical name to defined streaming processing pipeline, this will be used in the next step * Processing constraints, valid date to and from, define when this stream can execute * Event processing will use the event time provide within the received event * The use case will subscribe to events from the `nasdaq_quotes_stream` data source configured in step 1. * The use case applies 1 second tumbling window aggregate functions for two event attributes grouped by `symbol` * A simple select projection emits the computed events ### Example tumbling window calculations ```yaml stream: name: basic_tumbling_window_pipeline enabled: true eventTimeType: EVENT_TIME sources: - nasdaq_quotes_stream processing unit: pipeline: - time window: emitting type: tumblingQuoteAnalytics aggregate functions: FIRST: - ask LAST: - ask policy: type: tumblingTime window size: 1000 emit: select: symbol, ask_FIRST, ask_LAST group by: - symbol ``` ## 3. Distribute processed events Distribution of processed events can be as simple as to file, dashboard, or on to another streaming channel for another process to perform further processing. For this example we are using the Kafka sink connector. For further information on available sinks can be found [here](/joule/components/connectors/sinks.md). #### Overview of the definition * Provide a logical name for the distribution definition * Bind to the use case in this case it is `streamingAnalyticsPublisher` * Define one or more channels to receive events * The published event is created by mapping the internal Joule event to the domain type defined by the transform StockAnalyticRecordTransform implementation which will then be converted to Json ### Example Kafka publish connection using a custom transformation ```yaml kafkaPublisher: name: kafka_analytics_view cluster address: joule-gs-redpanda-0:9092 topic: analytics_view partitionKeys: - symbol serializer: transform: com.fractalworks.examples.banking.data.StockAnalyticRecordTransform key serializer: org.apache.kafka.common.serialization.IntegerSerializer value serializer: com.fractalworks.streams.transport.kafka.serializers.json.ObjectJsonSerializer ``` ## Deployment artefact Now we bring together each deployment artefact (source, use case and sinks) to form the desired use case. A use case is formed by a single app.env file which references these files. This method of deployment enables you to simply switch out the source and sinks based upon your needs i.e. development, testing and production deployments

use case file brings each descriptor file to form a single unit of deployment

### Binding file used to run a use case ```yaml use case: name: basic_twindow_analytics constraints: valid from: "2024-01-01T08:00:00.000Z" valid to: "2030-01-01T23:59:00.000Z" sources: - nasdaq_quotes_stream stream name: basic_tumbling_window_pipeline sinks: - kafka_analytics_view ``` --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.fractalworks.io/joule/getting-started/quickstart.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.