Event Tap

Joule provides the ability to tap an event stream in to an in-process / memory database. This feature is useful to gain data in SQL format for addtional purposes such as off-line analytic. Combining this feature with the provided Rest API functionality it becomes a powerful feature.
Example
tap:
  target schema: nasdaq_quotes
  queue capacity: 15000
  flush frequency: 5
  index:
    unique: false
    fields:
      - symbol
Attributes
Attribute
Description
Data Type
Required
target schema
Target table name
String
queue capacity
Number of events to queue before flushing to database.
​
Must be greater than 99.
Integer
Default :10000
flush frequency
Frequency the queue is flushed to database table. Either the queue capacity or this attribute triggers queue flush whatever comes first.
​
Must me greater than zereo.
Long
Default: 5 Seconds
alias
Alias to use instead of the schema name
String
is insert
Update or insert to apply to table changes
Boolean
Default: true
update criteria fields
Array of fields used for the update criteria predicate. Only used if is insert set to false
String[]
index
Create a table index
See Index attributes
Index Attributes
Attribute
Description
Data Type
Required
uniquie index
Create a unique index
Boolean
Default: false
index fields
Array of fields to create index
String[]
SQL Queries
On processor initialisation a set of queries are auto generated which are used to bind Rest API endpoints. The following queries are generated:
Select
Select data using pagnation either with or without an ingestTime date range.  
Export
Ability to export the table either by CSV or Parquet formats.
Delete
Query to delete rows from the table between an ingestTime date range. 
​

Attribute	Description	Data Type
target schema	Target table name	String
queue capacity	Number of events to queue before flushing to database. Must be greater than 99.	Integer Default :10000
flush frequency	Frequency the queue is flushed to database table. Either the queue capacity or this attribute triggers queue flush whatever comes first. Must me greater than zereo.	Long Default: 5 Seconds
alias	Alias to use instead of the schema name	String
is insert	Update or insert to apply to table changes	Boolean Default: true
update criteria fields	Array of fields used for the update criteria predicate. Only used if `is insert` set to false	String[]
index	Create a table index	See Index attributes

Attribute	Description	Data Type	Required
uniquie index	Create a unique index	Boolean Default: false
index fields	Array of fields to create index	String[]

Last modified 7d ago