MinIO S3

The storage flexibility of S3 provides the developer that ability to leverage existing data assets within a streaming context

---

Driver details io.minio:minio:8.5.4

Overview

This storage type is suited for very slow changing data which can be cached and used within a streaming context, therefore this data would have a low update frequency cycle.

Example configuration

minio stores:
    name: Telco ML Models
    connection:
      endpoint: "https://localhost"
      port: 9000
      tls: false
      credentials:
        access key: "XXXXXXX"
        secret key: "YYYYYYYYYYYYYYY"
    stores:
      predictors:
        bucketId: models
        initial version Id: 12345
        download dir: /home/joule/telco-models/tmp

This example uses the MinIO S3 connector to load a ML model on start up using the specified credentials and bucket details. The reference implementation is the pmml predictor processor, see documentation for further details.

Core attributes

Attribute	Description	Data Type	Required
name	Logical name of reference data set for given target connection	String
connection	Connection details	See Connection configuration
stores	Map of logical names to bucket level configurations. See bucket attributes section	Map

Bucket Attributes

Attribute	Description	Data Type	Required
bucketId	S3 bucket where the reference data resides	String
initial version id	The version id to use when priming the system. This is a global version for the given bucket otherwise the latest version of the object is used	String
download dir	Local path for bucket objects to be downloaded before processing starts	String Default: ./tmp