Joule
  • Welcome to Joule's Docs
  • Why Joule?
    • Joule capabilities
  • What is Joule?
    • Key features
    • The tech stack
  • Use case enablement
    • Use case building framework
  • Concepts
    • Core concepts
    • Low code development
    • Unified execution engine
    • Batch and stream processing
    • Continuous metrics
    • Key Joule data types
      • StreamEvent object
      • Contextual data
      • GeoNode
  • Tutorials
    • Getting started
    • Build your first use case
    • Stream sliding window quote analytics
    • Advanced tutorials
      • Custom missing value processor
      • Stateless Bollinger band analytics
      • IoT device control
  • FAQ
  • Glossary
  • Components
    • Pipelines
      • Use case anatomy
      • Data priming
        • Types of import
      • Processing unit
      • Group by
      • Emit computed events
      • Telemetry auditing
    • Processors
      • Common attributes
      • Filters
        • By type
        • By expression
        • Send on delta
        • Remove attributes
        • Drop all events
      • Enrichment
        • Key concepts
          • Anatomy of enrichment DSL
          • Banking example
        • Metrics
        • Dynamic contextual data
          • Caching architecture
        • Static contextual data
      • Transformation
        • Field Tokeniser
        • Obfuscation
          • Encryption
          • Masking
          • Bucketing
          • Redaction
      • Triggers
        • Change Data Capture
        • Business rules
      • Stream join
        • Inner stream joins
        • Outer stream joins
        • Join attributes & policy
      • Event tap
        • Anatomy of a Tap
        • SQL Queries
    • Analytics
      • Analytic tools
        • User defined analytics
          • Streaming analytics example
          • User defined analytics
          • User defined scripts
          • User defined functions
            • Average function library
        • Window analytics
          • Tumbling window
          • Sliding window
          • Aggregate functions
        • Analytic functions
          • Stateful
            • Exponential moving average
            • Rolling Sum
          • Stateless
            • Normalisation
              • Absolute max
              • Min max
              • Standardisation
              • Mean
              • Log
              • Z-Score
            • Scaling
              • Unit scale
              • Robust Scale
            • Statistics
              • Statistic summaries
              • Weighted moving average
              • Simple moving average
              • Count
            • General
              • Euclidean
        • Advanced analytics
          • Geospatial
            • Entity geo tracker
            • Geofence occupancy trigger
            • Geo search
            • IP address resolver
            • Reverse geocoding
            • Spatial Index
          • HyperLogLog
          • Distinct counter
      • ML inferencing
        • Feature engineering
          • Scripting
          • Scaling
          • Transform
        • Online predictive analytics
        • Model audit
        • Model management
      • Metrics engine
        • Create metrics
        • Apply metrics
        • Manage metrics
        • Priming metrics
    • Contextual data
      • Architecture
      • Configuration
      • MinIO S3
      • Apache Geode
    • Connectors
      • Sources
        • Kafka
          • Ingestion
        • RabbitMQ
          • Further RabbitMQ configurations
        • MQTT
          • Topic wildcards
          • Session management
          • Last Will and Testament
        • Rest endpoints
        • MinIO S3
        • File watcher
      • Sinks
        • Kafka
        • RabbitMQ
          • Further configurations
        • MQTT
          • Persistent messaging
          • Last Will and Testament
        • SQL databases
        • InfluxDB
        • MongoDB
        • Geode
        • WebSocket endpoint
        • MinIO S3
        • File transport
        • Slack
        • Email
      • Serialisers
        • Serialisation
          • Custom transform example
          • Formatters
        • Deserialisers
          • Custom parsing example
    • Observability
      • Enabling JMX for Joule
      • Meters
      • Metrics API
  • DEVELOPER GUIDES
    • Setting up developer environment
      • Environment setup
      • Build and deploy
      • Install Joule
        • Install Docker demo environment
        • Install with Docker
        • Install from source
        • Install Joule examples
    • Joulectl CLI
    • API Endpoints
      • Mangement API
        • Use case
        • Pipelines
        • Data connectors
        • Contextual data
      • Data access API
        • Query
        • Upload
        • WebSocket
      • SQL support
    • Builder SDK
      • Connector API
        • Sources
          • StreamEventParser API
        • Sinks
          • CustomTransformer API
      • Processor API
      • Analytics API
        • Create custom metrics
        • Define analytics
        • Windows API
        • SQL queries
      • Transformation API
        • Obfuscation API
        • FieldTokenizer API
      • File processing
      • Data types
        • StreamEvent
        • ReferenceDataObject
        • GeoNode
    • System configuration
      • System properties
  • Deployment strategies
    • Deployment Overview
    • Single Node
    • Cluster
    • GuardianDB
    • Packaging
      • Containers
      • Bare metal
  • Product updates
    • Public Roadmap
    • Release Notes
      • v1.2.0 Join Streams with stateful analytics
      • v1.1.0 Streaming analytics enhancements
      • v1.0.4 Predictive stream processing
      • v1.0.3 Contextual SQL based metrics
    • Change history
Powered by GitBook
On this page
  • Queue based eventing
  • Work Queues
  • Publish / Subscribe
  • Routing
  • Topic

Was this helpful?

  1. Components
  2. Connectors
  3. Sources
  4. RabbitMQ

Further RabbitMQ configurations

Further collection of examples using RabbitMQ

All examples assume default values when not applied

Queue based eventing

In this example, a simple RabbitMQ consumer is set up to subscribe directly to events from a queue. This approach is used for point-to-point event delivery, where each event is consumed by a single consumer from the queue.

Queue-based eventing ensures that each message is processed once by a consumer, providing reliable message handling.

rabbitConsumer:
  queue: point_to_point_queue
  ...

Work Queues

In this example, RabbitMQ is used to implement work queues, which distribute time-consuming tasks among multiple consumer workers.

This setup ensures that tasks are processed efficiently by a pool of workers, with each worker receiving a manageable number of tasks from the queue. It also provides fine-grained control over message acknowledgment and task distribution.

rabbitConsumer:
  queue: worker_queue
  autoAck: false
  durable: true
  prefetchCount: 1
  ...

Publish / Subscribe

In this example, RabbitMQ is configured to use the publish / subscribe pattern, where multiple consumer processes receive the same message. This is achieved using a fanout exchange, which broadcasts messages to all bound queues without requiring specific routing keys.

Since the fanout exchange type sends a copy of each message to all queues bound to it, there’s no need to specify routing keys. Every consumer connected to the exchange will receive the same event.

rabbitConsumer:
  exchange:
    name: quotes_exchange
    type: FANOUT    
  ...

Routing

In this example, RabbitMQ is configured to use direct routing, where each consumer subscribes only to a specific subset of messages based on the routing keys. This allows for more selective message consumption, meaning that the consumer will only receive the events it is interested in, based on its assigned routing keys.

  1. Exchange type The quotes_exchange is of type DIRECT, which routes messages based on exact matching of routing keys.

  2. Routing keys In this case, the consumer subscribes to two specific routing keys: NASDAQ.IBM and NASDAQ.MSFT. This means the consumer will only receive messages related to these two stock quotes.

rabbitConsumer:
  host: localhost
  exchange:
    name: quotes_exchange
    type: DIRECT

  routing:
    keys:
      - NASDAQ.IBM
      - NASDAQ.MSFT
  ...

Topic

In this example, RabbitMQ uses a topic exchange to allow consumers to subscribe to a wider set of events using wildcards.

The topic exchange type enables more flexible routing compared to direct exchanges, allowing the consumer to use patterns with the dot notation to match multiple routing keys.

  1. Exchange type The quotes_exchange is of type TOPIC, which allows the use of routing key patterns with wildcards (* and #).

  2. Routing keys The consumer subscribes to routing keys with wildcards to receive all quotes for specific stocks. In this case, *.IBM and *.MSFT will match all events for any market venue related to IBM and MSFT.

    1. The * wildcard matches exactly one word in the routing key.

    2. This allows the consumer to receive market venue quotes for both IBM and MSFT from any market source (e.g., NASDAQ, NYSE).

rabbitConsumer:
  host: localhost
  exchange:
    name: quotes_exchange
    type: TOPIC

  routing:
    keys:
      - "*.IBM"
      - "*.MSFT"
  ...
PreviousRabbitMQNextMQTT

Last updated 6 months ago

Was this helpful?