# Low code development
## What we will learn on this page
We will explore the Joule platform's low-code approach and its core features.
By the end of the article, we will have a clear understanding of how Joule simplifies development through its Domain-Specific Language (DSL) and the definition of use cases.
We will learn about:
1. **Joule low-code approach**\
Simplifying development using YAML-based DSL.
2. **Use case definition**\
Combining data sources, processing, and outputs into cohesive definitions.
3. **Streams**\
Configuring data processing pipelines, like tumbling windows.
4. **Data subscription & publishing**\
Connecting to external data sources and publishing results.
5. **Contextual data**\
Managing slower-changing data with in-memory caching for low-latency reads.
These concepts are introduced with high-level examples and can be explored in more detail with the linked documentation.
## Joule low-code approach
The Joule platform offers a low-code development solution that simplifies coding complexity by using a high-level language. This is accomplished through the Joule DSL, which enables the definition of use cases using human-readable YAML syntax.
{% hint style="success" %}
This forms the key platform **low code approach**
{% endhint %}
## Use case definition
A use case is defined by combining source data requirements, processing pipeline, and output destinations into a single cohesive definition.
### Example
The following diagram show the components of a use case. Each use case dependency is linked using a logical name which exists within an independent file.
Use case components
This results in a single use case definition:
```yaml
use case:
name: nasdaq_buy_signaler
constraints:
valid from: "2024-01-01T08:00:00.000Z"
valid to: "2030-01-01T23:59:00.000Z" # empty for infinite processing
sources:
- live_nasdaq_quotes
stream name: quote_buy_signals
sinks:
- client_buy_dashboards
```
## Stream
A stream defines the actual processing requirements and sequence.
{% content-ref url="../components/pipelines" %}
[pipelines](https://docs.fractalworks.io/joule/components/pipelines)
{% endcontent-ref %}
### Example
The below example creates the `min` & `max` for `ask` & `bid` values within a five-second tumbling window and only publishes symbols where they are not `'A'`.
```yaml
stream:
name: basic_tumbling_window_pipeline
eventTimeType: EVENT_TIME
sources:
- nasdaq_quotes_stream
processing unit:
pipeline:
- time window:
emitting type: tumblingQuoteAnalytics
aggregate functions:
MIN: [ask, bid]
MAX: [ask, bid]
policy:
type: tumblingTime
windowSize: 5000
emit:
eventType: windowQuoteEvent
select: "symbol, ask_MIN, ask_MAX, bid_MIN, bid_MAX"
having: "symbol !='A'"
group by:
- symbol
```
## Data subscription
Users are able to subscribe to external data events through the use of source connectors.
{% content-ref url="../components/connectors/sources" %}
[sources](https://docs.fractalworks.io/joule/components/connectors/sources)
{% endcontent-ref %}
### Example
The below example connects to a Kafka cluster, consumes events from the quote topic and transforms the received quote object into an internal `StreamEvent` object.
```yaml
consumer:
name: nasdaq_quotes_stream
sources:
- kafkaConsumer:
name: nasdaq_quotes_stream
cluster address: KAFKA_BROKER:19092
consumerGroupId: nasdaq
topics:
- quotes
deserializer:
parser: com.fractalworks.examples.banking.data.QuoteToStreamEventParser
key deserializer: org.apache.kafka.common.serialization.IntegerDeserializer
value deserializer: com.fractalworks.streams.transport.kafka.serializers.object.ObjectDeserializer
```
## Event publishing
Users are able to publish events to downstream data platforms through the use of destination connectors.
{% content-ref url="../components/connectors/sinks" %}
[sinks](https://docs.fractalworks.io/joule/components/connectors/sinks)
{% endcontent-ref %}
### Example
The below example generates a `quoteWindowStream.csv` file from the `tumblingWindowQuoteStream` events.
```yaml
publisher:
name: standardAnalyticsFilePublisher
source: basic_tumbling_window_pipeline
sinks:
- file:
enabled: true
filename: nasdaqAnalytic
path: ./data/output/analytics
batchSize: 1024
timeout: 1000
formatter:
csv formatter:
contentType: "text/csv"
encoding: "UTF-8"
delimiter: "|"
```
## Contextual data
Often in stream processing additional data is required to perform analytics, generally known as reference or contextual data.
Data of this form generally updates at a much slower pace and therefore is managed differently and held in data platform not architected for low latency reads. Joule has build a low latency read mechanism to overcome this limitation using in-memory caching.
{% content-ref url="../components/contextual-data" %}
[contextual-data](https://docs.fractalworks.io/joule/components/contextual-data)
{% endcontent-ref %}
### Example
The below example connects to a distributed caching platform, [Apache Geode](https://geode.apache.org), for a low latency reference data reads.
```yaml
reference data:
name: banking market data
data sources:
- geode stores:
name: us markets
connection:
locator address: 192.168.86.39
locator port: 41111
stores:
nasdaqIndexCompanies:
region: nasdaq-companies
keyClass : java.lang.String
gii: true
holidays:
region: us-holidays
keyClass : java.lang.Integer
```
