Model audit

Support explainability, detect drift, and manage retraining through model auditing

Objective

This optional configuration provide the ability to audit predications to enable model retraining, feature and prediction drift management, model observability and any local business governance requirements.

The configuration will dynamically create an in-memory database table, using the process name as the target table and rest endpoints to enable direct access and export functions.

Use this feature to support external drift monitoring and model retraining

Example & DSL attributes

For more details or assistance, Fractalworks is ready to support in your enquiries

This configuration enables the auditing of model predictions for monitoring and retraining purposes.

The pmml predictor is configured to use a specific model: iris_rf.pmml and the audit configuration sets up an in-memory database to store audit data such as predictions.

The schema is set to ml_audit, with a queue capacity of 5000 and a flush frequency of 300 seconds, allowing for real-time tracking of prediction performance and potential model drift.

Example

pmml predictor:
  name: irisScorer
  model: ./models/iris_rf.pmml
  response field: flowerPrediction
  
  audit configuration:
    target schema: ml_audit
    queue capacity: 5000
    flush frequency: 300

Attributes schema

Attribute	Description	Data Type	Required
target schema	name of the target schema where the table will be created	String
queue capacity	Number of prediction results to queue before flushing to target table	Integer Default: 1000
flush frequency	Frequency the queue will be flushed to table as seconds	Long Default: 60 Seconds