Event Tap
Joule provides the ability to tap an event stream in to an in-process / memory database. This feature is useful to gain data in SQL format for addtional purposes such as off-line analytic. Combining this feature with the provided Rest API functionality it becomes a powerful feature.
tap:
target schema: nasdaq_quotes
queue capacity: 15000
flush frequency: 5
index:
unique: false
fields:
- symbol
Attribute | Description | Data Type | Required |
---|---|---|---|
target schema | Target table name | String | |
queue capacity | Number of events to queue before flushing to database. Must be greater than 99. | Integer Default :10000 | |
flush frequency | Frequency the queue is flushed to database table. Either the queue capacity or this attribute triggers queue flush whatever comes first. Must me greater than zereo. | Long Default: 5 Seconds | |
alias | Alias to use instead of the schema name | String | |
is insert | Update or insert to apply to table changes | Boolean Default: true | |
update criteria fields | Array of fields used for the update criteria predicate. Only used if is insert set to false | String[] | |
index | Create a table index | See Index attributes |
Attribute | Description | Data Type | Required |
---|---|---|---|
uniquie index | Create a unique index | Boolean Default: false | |
index fields | Array of fields to create index | String[] |
On processor initialisation a set of queries are auto generated which are used to bind Rest API endpoints. The following queries are generated:
Select data using pagnation either with or without an ingestTime date range.
Ability to export the table either by CSV or Parquet formats.
Query to delete rows from the table between an ingestTime date range.
Last modified 7d ago