Joule
  • Welcome to Joule's Docs
  • Why Joule?
    • Joule capabilities
  • What is Joule?
    • Key features
    • The tech stack
  • Use case enablement
    • Use case building framework
  • Concepts
    • Core concepts
    • Low code development
    • Unified execution engine
    • Batch and stream processing
    • Continuous metrics
    • Key Joule data types
      • StreamEvent object
      • Contextual data
      • GeoNode
  • Tutorials
    • Getting started
    • Build your first use case
    • Stream sliding window quote analytics
    • Advanced tutorials
      • Custom missing value processor
      • Stateless Bollinger band analytics
      • IoT device control
  • FAQ
  • Glossary
  • Components
    • Pipelines
      • Use case anatomy
      • Data priming
        • Types of import
      • Processing unit
      • Group by
      • Emit computed events
      • Telemetry auditing
    • Processors
      • Common attributes
      • Filters
        • By type
        • By expression
        • Send on delta
        • Remove attributes
        • Drop all events
      • Enrichment
        • Key concepts
          • Anatomy of enrichment DSL
          • Banking example
        • Metrics
        • Dynamic contextual data
          • Caching architecture
        • Static contextual data
      • Transformation
        • Field Tokeniser
        • Obfuscation
          • Encryption
          • Masking
          • Bucketing
          • Redaction
      • Triggers
        • Change Data Capture
        • Business rules
      • Stream join
        • Inner stream joins
        • Outer stream joins
        • Join attributes & policy
      • Event tap
        • Anatomy of a Tap
        • SQL Queries
    • Analytics
      • Analytic tools
        • User defined analytics
          • Streaming analytics example
          • User defined analytics
          • User defined scripts
          • User defined functions
            • Average function library
        • Window analytics
          • Tumbling window
          • Sliding window
          • Aggregate functions
        • Analytic functions
          • Stateful
            • Exponential moving average
            • Rolling Sum
          • Stateless
            • Normalisation
              • Absolute max
              • Min max
              • Standardisation
              • Mean
              • Log
              • Z-Score
            • Scaling
              • Unit scale
              • Robust Scale
            • Statistics
              • Statistic summaries
              • Weighted moving average
              • Simple moving average
              • Count
            • General
              • Euclidean
        • Advanced analytics
          • Geospatial
            • Entity geo tracker
            • Geofence occupancy trigger
            • Geo search
            • IP address resolver
            • Reverse geocoding
            • Spatial Index
          • HyperLogLog
          • Distinct counter
      • ML inferencing
        • Feature engineering
          • Scripting
          • Scaling
          • Transform
        • Online predictive analytics
        • Model audit
        • Model management
      • Metrics engine
        • Create metrics
        • Apply metrics
        • Manage metrics
        • Priming metrics
    • Contextual data
      • Architecture
      • Configuration
      • MinIO S3
      • Apache Geode
    • Connectors
      • Sources
        • Kafka
          • Ingestion
        • RabbitMQ
          • Further RabbitMQ configurations
        • MQTT
          • Topic wildcards
          • Session management
          • Last Will and Testament
        • Rest endpoints
        • MinIO S3
        • File watcher
      • Sinks
        • Kafka
        • RabbitMQ
          • Further configurations
        • MQTT
          • Persistent messaging
          • Last Will and Testament
        • SQL databases
        • InfluxDB
        • MongoDB
        • Geode
        • WebSocket endpoint
        • MinIO S3
        • File transport
        • Slack
        • Email
      • Serialisers
        • Serialisation
          • Custom transform example
          • Formatters
        • Deserialisers
          • Custom parsing example
    • Observability
      • Enabling JMX for Joule
      • Meters
      • Metrics API
  • DEVELOPER GUIDES
    • Setting up developer environment
      • Environment setup
      • Build and deploy
      • Install Joule
        • Install Docker demo environment
        • Install with Docker
        • Install from source
        • Install Joule examples
    • Joulectl CLI
    • API Endpoints
      • Mangement API
        • Use case
        • Pipelines
        • Data connectors
        • Contextual data
      • Data access API
        • Query
        • Upload
        • WebSocket
      • SQL support
    • Builder SDK
      • Connector API
        • Sources
          • StreamEventParser API
        • Sinks
          • CustomTransformer API
      • Processor API
      • Analytics API
        • Create custom metrics
        • Define analytics
        • Windows API
        • SQL queries
      • Transformation API
        • Obfuscation API
        • FieldTokenizer API
      • File processing
      • Data types
        • StreamEvent
        • ReferenceDataObject
        • GeoNode
    • System configuration
      • System properties
  • Deployment strategies
    • Deployment Overview
    • Single Node
    • Cluster
    • GuardianDB
    • Packaging
      • Containers
      • Bare metal
  • Product updates
    • Public Roadmap
    • Release Notes
      • v1.2.0 Join Streams with stateful analytics
      • v1.1.0 Streaming analytics enhancements
      • v1.0.4 Predictive stream processing
      • v1.0.3 Contextual SQL based metrics
    • Change history
Powered by GitBook
On this page

Was this helpful?

  1. DEVELOPER GUIDES
  2. API Endpoints
  3. Data access API

Query

Fetch, export, delete and subscribe to data

PreviousData access APINextUpload

Last updated 7 months ago

Was this helpful?


Overview

The Query API allows developers to list available data to query, fetch, export and delete data from the internal in-memory database.


Get list of available table queries to execute

get
Responses
200
OK
get
GET /joule/query/list HTTP/1.1
Host: 
Accept: */*
200

OK

No content

Fetch events by offset and page size

get
Path parameters
schemastringRequired

Database schema where the table resides. Dynamically set by the use case DSL as the stream processing name

Example: standardQuoteAnalyticsStream
tablestringRequired

Table to trigger data export

Example: quotes
Query parameters
fromstringOptional

Date time parameter to define point in time to start from.

Example: 2022-10-14 17:00:00
tostringOptional

Date time parameter to define point in time to go too.

Example: 2022-10-14 18:00:00
offsetinteger · int32Required

Page offset to fetch next set of events.

Example: 1
pagesizeinteger · int32Optional

Number of events to return.

Example: 1000
Responses
200
OK
application/json
400
Missing or invalid parameters
500
Internal Joule error. Check log files.
get
GET /joule/query/{schema}/{table}?offset=1 HTTP/1.1
Host: 
Accept: */*
[
  {
    "eventType": "text",
    "lineage": {
      "ANY_ADDITIONAL_PROPERTY": 1
    },
    "eventTime": 1,
    "ingestTime": 1,
    "value": {},
    "uUID": "text",
    "subType": {},
    "dictionary": {
      "ANY_ADDITIONAL_PROPERTY": {}
    },
    "snapshot": {
      "": {},
      "present": true,
      "empty": true
    }
  }
]

Trigger export table process using date time constraint.

get
Path parameters
schemastringRequired

Database schema where the table resides. Dynamically set by the use case DSL as the stream processing name.

Example: standardQuoteAnalyticsStream
tablestringRequired

Table to trigger data export upon.

Example: processed
typestringRequired

The data export type. Supported types parquet and csv.

Example: parquet
Query parameters
fromstringRequired

Date time parameter to define point in time to start from.

Example: 2022-10-14 17:00:00
tostringRequired

Date time parameter to define point in time to go too.

Example: 2022-10-14 18:00:00
pathstringRequired

Place to land export file too

Example: ~/quotes.parquet
Responses
200
OK
400
Missing or invalid parameters
500
Internal Joule error. Check log files.
get
GET /joule/export/{schema}/{table}/{type}?from=2022-10-14+17%3A00%3A00&to=2022-10-14+18%3A00%3A00&path=~%2Fquotes.parquet HTTP/1.1
Host: 
Accept: */*

No content

Trigger table delete process using date time constraints.

delete
Path parameters
schemastringRequired

Database schema where the table resides. Dynamically set by the use case DSL as the stream processing name.

Example: standardQuoteAnalyticsStream
tablestringRequired

Table to trigger data delete upon.

Example: processed
Query parameters
fromstringRequired

Date time parameter to define point in time to start from.

Example: 2022-10-14 17:00:00
tostringRequired

Date time parameter to define point in time to go too.

Example: 2022-10-14 18:00:00
Responses
200
OK
400
Missing or invalid parameters
500
Internal Joule error. Check log files.
delete
DELETE /joule/delete/{schema}/{table}?from=2022-10-14+17%3A00%3A00&to=2022-10-14+18%3A00%3A00 HTTP/1.1
Host: 
Accept: */*

No content

  • Overview
  • GETGet list of available table queries to execute
  • GETFetch events by offset and page size
  • GETTrigger export table process using date time constraint.
  • DELETETrigger table delete process using date time constraints.