# Components - [Pipelines](https://docs.fractalworks.io/joule/components/pipelines.md): Data (stream) pipelines enable business-specific, real-time analytics use cases - [Use case anatomy](https://docs.fractalworks.io/joule/components/pipelines/use-case-anatomy.md): A use case binds together the required data and stream pipeline unit to perform the desired function - [Data priming](https://docs.fractalworks.io/joule/components/pipelines/data-priming.md): Prime Joule with necessary startup data - [Types of import](https://docs.fractalworks.io/joule/components/pipelines/data-priming/types-of-import.md): Parquet and CSV data imports for stream processing - [Processing unit](https://docs.fractalworks.io/joule/components/pipelines/processing-unit.md): Quickly build custom business use cases - [Group by](https://docs.fractalworks.io/joule/components/pipelines/group-by.md): Groups events by attributes to drive optimised computation functions - [Emit computed events](https://docs.fractalworks.io/joule/components/pipelines/emit-computed-events.md): Select and filter output events for publishing - [Telemetry auditing](https://docs.fractalworks.io/joule/components/pipelines/telemetry-auditing.md): Audit and track events for validation and testing - [Processors](https://docs.fractalworks.io/joule/components/processors.md): Processors are the core of the Joule platform, each performing a specific task. These create use case when linked together - [Common attributes](https://docs.fractalworks.io/joule/components/processors/common-attributes.md): Core processor attributes that provide out-of-the-box features - [Filters](https://docs.fractalworks.io/joule/components/processors/filters.md): Reduce stream processing overhead by filtering irrelevant events - [By type](https://docs.fractalworks.io/joule/components/processors/filters/by-type.md): Filter events based on specific types from the processing pipeline - [By expression](https://docs.fractalworks.io/joule/components/processors/filters/by-expression.md): Use Javascript expressions to define custom filters - [Send on delta](https://docs.fractalworks.io/joule/components/processors/filters/send-on-delta.md): Stateful filter processor that filters previous seen events - [Remove attributes](https://docs.fractalworks.io/joule/components/processors/filters/remove-attributes.md): Remove sensitive attributes from an event - [Drop all events](https://docs.fractalworks.io/joule/components/processors/filters/drop-all-events.md): Drop all incoming events in the processor - [Enrichment](https://docs.fractalworks.io/joule/components/processors/enrichment.md): Enrich events using connected contextual data, metrics and analytics - [Key concepts](https://docs.fractalworks.io/joule/components/processors/enrichment/key-concepts.md): Enriching events with the latest contextual data is crucial for processing advanced real-time business insights - [Anatomy of enrichment DSL](https://docs.fractalworks.io/joule/components/processors/enrichment/key-concepts/anatomy-of-enrichment-dsl.md): The enrichment DSL provides a flexible syntax that enables the user to define field level enrichments - [Banking example](https://docs.fractalworks.io/joule/components/processors/enrichment/key-concepts/banking-example.md): The below banking example enriches events using the linked contextual data and metrics - [Enrichment catalogue](https://docs.fractalworks.io/joule/components/processors/enrichment/key-concepts/enrichment-catalogue.md) - [Old version of Key concepts](https://docs.fractalworks.io/joule/components/processors/enrichment/old-version-of-key-concepts.md): Enriching event data with the latest contextual data is crucial for processing real-time business insights. - [Metrics](https://docs.fractalworks.io/joule/components/processors/enrichment/metrics.md): Enrich events with pre-calculated metrics to facilitate further advanced processing - [Dynamic contextual data](https://docs.fractalworks.io/joule/components/processors/enrichment/dynamic-contextual-data.md): Use fast, connected data stores to quickly enrich in-motion events with up-to-date context - [Caching architecture](https://docs.fractalworks.io/joule/components/processors/enrichment/dynamic-contextual-data/caching-architecture.md): Caching is a key enabler for low-latency contextual data access - [Static contextual data](https://docs.fractalworks.io/joule/components/processors/enrichment/static-contextual-data.md): Enrich events with essential slow changing data from Joule’s in-memory database. - [Transformation](https://docs.fractalworks.io/joule/components/processors/transformation.md): Reshape origin event data in to target event - [Field Tokeniser](https://docs.fractalworks.io/joule/components/processors/transformation/field-tokeniser.md): Tokenise attribute values in to component parts - [Obfuscation](https://docs.fractalworks.io/joule/components/processors/transformation/obfuscation.md): Joule provides an in-stream-obfruscation function to streaming events. This function is useful for Personal Identifiable Information de-identification. - [Encryption](https://docs.fractalworks.io/joule/components/processors/transformation/obfuscation/encryption.md): Apply RSA encryption techniques along with AES symmetric encryption - [Masking](https://docs.fractalworks.io/joule/components/processors/transformation/obfuscation/masking.md): Mask sensitive field values while retaining the formatting criteria - [Bucketing](https://docs.fractalworks.io/joule/components/processors/transformation/obfuscation/bucketing.md): Apply individual variance tolerances to protect the identity of the originating data - [Redaction](https://docs.fractalworks.io/joule/components/processors/transformation/obfuscation/redaction.md): Eliminate sensitive event field data by either nullifying or replacing with a blank value - [Triggers](https://docs.fractalworks.io/joule/components/processors/triggers.md): Minimise expensive computation while notifying consumers of events to action - [Change Data Capture](https://docs.fractalworks.io/joule/components/processors/triggers/change-data-capture.md): Distribute delta records for key attribute changes - [Business rules](https://docs.fractalworks.io/joule/components/processors/triggers/business-rules.md): Leverage standard JSR-94 rules engine to perform advance business rules - [Stream join](https://docs.fractalworks.io/joule/components/processors/stream-join.md): Join independent stream events to trigger advanced analytics and dynamic business rules - [Inner stream joins](https://docs.fractalworks.io/joule/components/processors/stream-join/inner-stream-joins.md): Strict join between two event streams - [Outer stream joins](https://docs.fractalworks.io/joule/components/processors/stream-join/outer-stream-joins.md): Immediately pass the first event received and initialised downstream processors - [Join attributes & policy](https://docs.fractalworks.io/joule/components/processors/stream-join/join-attributes-and-policy.md): Join attributes to control and optimise performance - [Event tap](https://docs.fractalworks.io/joule/components/processors/event-tap.md): Tap events directly in to an in-memory database to enable on / offline processing - [Anatomy of a Tap](https://docs.fractalworks.io/joule/components/processors/event-tap/anatomy-of-a-tap.md): Construction of a Tap - [SQL Queries](https://docs.fractalworks.io/joule/components/processors/event-tap/sql-queries.md): Use SQL queries to access the data in the store - [Analytics](https://docs.fractalworks.io/joule/components/analytics.md): Analytics sit at the heart of the Joule platform, transforming raw data into actionable insights that drive real-world value - [Analytic tools](https://docs.fractalworks.io/joule/components/analytics/analytic-tools.md): Define math expressions or provide as a file using Joule supported languages and APIs - [User defined analytics](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/user-defined-analytics.md): Define analytic expressions or provide as a script using supported interpreted languages - [Streaming analytics example](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/user-defined-analytics/streaming-analytics-example.md): Example on how to run streaming analytics - [User defined analytics](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/user-defined-analytics/user-defined-analytics.md): Declarative stateful math expressions evaluated in real-time - [User defined scripts](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/user-defined-analytics/user-defined-scripts.md): Leverage pre-existing analytics scripts within a streaming context - [User defined functions](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/user-defined-analytics/user-defined-functions.md): Extend the analytics ecosystem using the Analytics API - [Average function library](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/user-defined-analytics/user-defined-functions/average-function-library.md): Joule math functions - [Window analytics](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/window-analytics.md): Standard window functions are provided to perform event based analytics - [Tumbling window](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/window-analytics/tumbling-window.md): Fixed-sized, non-overlapping window analytic function support - [Sliding window](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/window-analytics/sliding-window.md): Fixed-sized, overlapping time or event count window analytic function support - [Aggregate functions](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/window-analytics/aggregate-functions.md): Standard statistics calculations for streaming event windows - [Analytic functions](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions.md): Analytic functions perform complex calculation over a window of grouped events - [Stateful](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateful.md): Stateful based analytics over a grouped set of events - [Exponential moving average](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateful/exponential-moving-average.md) - [Rolling Sum](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateful/rolling-sum.md) - [Stateless](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless.md): Stateless based analytics over a grouped set of events - [Normalisation](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/normalisation.md): Change the distribution shape of your data - [Absolute max](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/normalisation/absolute-max.md) - [Min max](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/normalisation/min-max.md) - [Standardisation](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/normalisation/standardisation.md) - [Mean](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/normalisation/mean.md) - [Log](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/normalisation/log.md) - [Z-Score](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/normalisation/z-score.md) - [Scaling](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/scaling.md): Change the range of your data - [Unit scale](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/scaling/unit-scale.md) - [Robust Scale](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/scaling/robust-scale.md) - [Statistics](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/statistics.md): Key statistical functions and measures - [Statistic summaries](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/statistics/statistic-summaries.md) - [Weighted moving average](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/statistics/weighted-moving-average.md) - [Simple moving average](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/statistics/simple-moving-average.md) - [Count](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/statistics/count.md) - [General](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/general.md): Utility and supporting analytic functions - [Euclidean](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/analytic-functions/stateless/general/euclidean.md) - [Advanced analytics](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/advanced-analytics.md): Advanced analytic processors such as geospatial tracking - [Geospatial](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/advanced-analytics/geospatial.md): Optimised real-time geospatial analytics for geofence occupancy and tracking - [Entity geo tracker](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/advanced-analytics/geospatial/entity-geo-tracker.md): Real-time entity geo tracking for advanced use cases - [Geofence occupancy trigger](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/advanced-analytics/geospatial/geofence-occupancy-trigger.md): Trigger geospatial events in real-time to drive location based use cases - [Geo search](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/advanced-analytics/geospatial/geo-search.md): Real-time view on what entities are nearby with respect to geofence radius - [IP address resolver](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/advanced-analytics/geospatial/ip-address-resolver.md): Take an IP address and perform a geo location search - [Reverse geocoding](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/advanced-analytics/geospatial/reverse-geocoding.md): Google reverse geocode using longitude and latitude to street address - [Spatial Index](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/advanced-analytics/geospatial/spatial-index.md): Optimised data structure for spatial search - [HyperLogLog](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/advanced-analytics/hyperloglog.md): Probabilistic counter for large or high cardinality datasets - [Distinct counter](https://docs.fractalworks.io/joule/components/analytics/analytic-tools/advanced-analytics/distinct-counter.md): Calculate distinct field values over time - [ML inferencing](https://docs.fractalworks.io/joule/components/analytics/ml-inferencing.md): Leverage streaming online predictions to drive insights to action - [Key concepts](https://docs.fractalworks.io/joule/components/analytics/ml-inferencing/key-concepts.md) - [Feature engineering](https://docs.fractalworks.io/joule/components/analytics/ml-inferencing/feature-engineering.md): Decorate a feature vector with enriched features specific to the deployed model - [Scripting](https://docs.fractalworks.io/joule/components/analytics/ml-inferencing/feature-engineering/scripting.md): Define custom analytics with declarative expressions - [Scaling](https://docs.fractalworks.io/joule/components/analytics/ml-inferencing/feature-engineering/scaling.md): Normalise data with various scaling methods - [Transform](https://docs.fractalworks.io/joule/components/analytics/ml-inferencing/feature-engineering/transform.md): Generate analytics-ready features from data - [Online predictive analytics](https://docs.fractalworks.io/joule/components/analytics/ml-inferencing/online-predictive-analytics.md): JPMML prediction processor evaluates event feature vectors in near-real-time - [Model audit](https://docs.fractalworks.io/joule/components/analytics/ml-inferencing/model-audit.md): Support explainability, detect drift, and manage retraining through model auditing - [Model management](https://docs.fractalworks.io/joule/components/analytics/ml-inferencing/model-management.md): Deploy a retrained model directly in to a running Joule with zero down time - [Metrics engine](https://docs.fractalworks.io/joule/components/analytics/metrics-engine.md): In-memory SQL compliant metrics engine for advance analytics solutions - [Create metrics](https://docs.fractalworks.io/joule/components/analytics/metrics-engine/create-metrics.md): Define a set of time based metrics that are generated using captured and stored streamed events - [Apply metrics](https://docs.fractalworks.io/joule/components/analytics/metrics-engine/apply-metrics.md): Apply metrics within stream processing functions - [Manage metrics](https://docs.fractalworks.io/joule/components/analytics/metrics-engine/manage-metrics.md): Reuse existing metrics on startup and reduce the operational memory overhead - [Priming metrics](https://docs.fractalworks.io/joule/components/analytics/metrics-engine/priming-metrics.md): Prime metrics with existing values on process startup - [Exporting metrics](https://docs.fractalworks.io/joule/components/analytics/metrics-engine/exporting-metrics.md) - [Contextual data](https://docs.fractalworks.io/joule/components/contextual-data.md): Enrich real-time streams with contextual data - [Architecture](https://docs.fractalworks.io/joule/components/contextual-data/architecture.md): Understand how Joule integrates to contextual data solutions - [Configuration](https://docs.fractalworks.io/joule/components/contextual-data/configuration.md): Set up a contextual data-driven use case - [MinIO S3](https://docs.fractalworks.io/joule/components/contextual-data/minio-s3.md): Multi-cloud S3-compatible interface for static contextual data and ML models - [Apache Geode](https://docs.fractalworks.io/joule/components/contextual-data/apache-geode.md): High-performance caching platform for static and slow-moving contextual data - [Connectors](https://docs.fractalworks.io/joule/components/connectors.md): Out-of-the-box Joule provides key data connectors for streaming use cases - [Sources](https://docs.fractalworks.io/joule/components/connectors/sources.md): Connect to key event and file data sources - [Kafka](https://docs.fractalworks.io/joule/components/connectors/sources/kafka.md): Standard Kafka consumers transport ingests data from subscribed cluster topics - [Ingestion](https://docs.fractalworks.io/joule/components/connectors/sources/kafka/ingestion.md): Joule enables flexible Kafka data deserialisation for processing - [RabbitMQ](https://docs.fractalworks.io/joule/components/connectors/sources/rabbitmq.md): AMQP messaging solution ideal for IoT and client / server use cases - [Further RabbitMQ configurations](https://docs.fractalworks.io/joule/components/connectors/sources/rabbitmq/further-rabbitmq-configurations.md): Further collection of examples using RabbitMQ - [MQTT](https://docs.fractalworks.io/joule/components/connectors/sources/mqtt.md): Lightweight messaging protocol ideal for IoT use cases - [Topic wildcards](https://docs.fractalworks.io/joule/components/connectors/sources/mqtt/topic-wildcards.md) - [Session management](https://docs.fractalworks.io/joule/components/connectors/sources/mqtt/session-management.md) - [Last Will and Testament](https://docs.fractalworks.io/joule/components/connectors/sources/mqtt/last-will-and-testament.md) - [Rest endpoints](https://docs.fractalworks.io/joule/components/connectors/sources/rest-endpoints.md): RESTful consumer endpoints designed to facilitate seamless integration - [MinIO S3](https://docs.fractalworks.io/joule/components/connectors/sources/minio-s3.md): MinIO S3 file consumer for cloud or local hosted bucket storage - [Google sheets](https://docs.fractalworks.io/joule/components/connectors/sources/google-sheets.md): Spreadsheet data consumer designed to extend processing capabilites - [File watcher](https://docs.fractalworks.io/joule/components/connectors/sources/file-watcher.md): Process large files using stream processing - [Sinks](https://docs.fractalworks.io/joule/components/connectors/sinks.md): Distribute insights to consuming systems - [Kafka](https://docs.fractalworks.io/joule/components/connectors/sinks/kafka.md): Sends events to specified Kafka topics, enabling real-time data streams - [RabbitMQ](https://docs.fractalworks.io/joule/components/connectors/sinks/rabbitmq.md): RabbitMQ is lightweight and easy to deploy messaging platform for event-based data integration - [Further configurations](https://docs.fractalworks.io/joule/components/connectors/sinks/rabbitmq/further-configurations.md): Further RabbitMQ distribution configurations - [MQTT](https://docs.fractalworks.io/joule/components/connectors/sinks/mqtt.md): Allows publishing events through MQTT, ideal for IoT and lightweight streaming - [Persistent messaging](https://docs.fractalworks.io/joule/components/connectors/sinks/mqtt/persistent-messaging.md) - [Last Will and Testament](https://docs.fractalworks.io/joule/components/connectors/sinks/mqtt/last-will-and-testament.md) - [SQL databases](https://docs.fractalworks.io/joule/components/connectors/sinks/sql-databases.md): Write events in micro batches to any database which supports JDBC Type 4 drivers - [InfluxDB](https://docs.fractalworks.io/joule/components/connectors/sinks/influxdb.md): Standard time-series database idea for instrumentation or live KPI tracking - [MongoDB](https://docs.fractalworks.io/joule/components/connectors/sinks/mongodb.md): Publish JSON based events to the mature standard document storePublishes JSON-based events to MongoDB, a widely-used document store - [Geode](https://docs.fractalworks.io/joule/components/connectors/sinks/geode.md): Integrate with distributed caching for complex use cases needing high availability - [WebSocket](https://docs.fractalworks.io/joule/components/connectors/sinks/websocket.md): Publishes processed events to web clients via WebSocket transport, ideal for real-time dashboards and applications - [Google sheets](https://docs.fractalworks.io/joule/components/connectors/sinks/google-sheets.md): Spreadsheet data publisher designed to extend processing capabilites - [Console](https://docs.fractalworks.io/joule/components/connectors/sinks/console.md): For testing purposes a console output processor is provided. - [MinIO S3](https://docs.fractalworks.io/joule/components/connectors/sinks/minio-s3.md): MinIO file producer using S3 cloud or local hosted buckets - [Custom connectors](https://docs.fractalworks.io/joule/components/connectors/sinks/custom-connectors.md) - [File](https://docs.fractalworks.io/joule/components/connectors/sinks/custom-connectors/file.md): Write events directly to files for archiving and offline processing - [Slack](https://docs.fractalworks.io/joule/components/connectors/sinks/custom-connectors/slack.md): Send critical alerts to a monitored Slack channel - [Email](https://docs.fractalworks.io/joule/components/connectors/sinks/custom-connectors/email.md): Send critical email alerts to key stakeholders - [Serialisers](https://docs.fractalworks.io/joule/components/connectors/serialisers.md): Get data ready for consumption serialising incoming and outgoing data - [Serialisation](https://docs.fractalworks.io/joule/components/connectors/serialisers/serialisation.md): Data emitted to downstream systems is serialised using provided or custom serialisers - [Custom transform example](https://docs.fractalworks.io/joule/components/connectors/serialisers/serialisation/custom-transform-example.md): Custom parser converts an internal Joule StreamEvent into a domain object - [Formatters](https://docs.fractalworks.io/joule/components/connectors/serialisers/serialisation/formatters.md): Apply standard data format process to outgoing data - [Deserialisers](https://docs.fractalworks.io/joule/components/connectors/serialisers/deserialisers.md): Consumed data is mapped to internal StreamEvents using a flexible deserialisation framework - [Custom parsing example](https://docs.fractalworks.io/joule/components/connectors/serialisers/deserialisers/custom-parsing-example.md): Custom parser converts a domain Quote object into an internal Joule StreamEvent - [Observability](https://docs.fractalworks.io/joule/components/observability.md): Joule uses JMX and Metrics API to make processes observable and to enable monitoring - [Enabling JMX for Joule](https://docs.fractalworks.io/joule/components/observability/enabling-jmx-for-joule.md): Activate JMX monitoring for Joule with configurable settings - [Meters](https://docs.fractalworks.io/joule/components/observability/meters.md): Automated metrics monitor for Joule pipelines: data pipelines, processors, transport, storage - [Metrics API](https://docs.fractalworks.io/joule/components/observability/metrics-api.md): Tracks component events and data flow with counters --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.fractalworks.io/joule/components.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.