Feature engineering
Decorate a feature vector with enriched features specific to the deployed model
“Coming up with features is difficult, time-consuming, requires expert knowledge. 'Applied machine learning' is basically feature engineering.” — Prof. Andrew Ng.
Objective
Joule provides a feature engineering processor that enables users to define how features are to be created ready for predictive analytics use cases.
The processor generates for each declared feature field an engineered value. Two methods are supported:
raw
compute values using custom expression and plugins
On completion a feature map is generated with all the required features and placed in the StreamEvent
ready for the next processor in the pipeline.
Example
Attributes schema
Top level attributes
Attribute | Description | Data Type | Required |
---|---|---|---|
name | Name feature set which is used for a predicting model | String | |
versioned | A boolean flag to apply a unique version identifier to the resulting feature map | Boolean Default: true | |
features | List of supported feature functions | List |
The features attribute provide two key elements, as value
and compute
. Either one of the attributes must be defined.
Attributes schema
Attribute | Description | Data Type | Required |
---|---|---|---|
as values | List of event fields whose value will be copied in to the feature map without any changes | List | |
compute | List of supported feature functions mapped to output variables to be executed using the passed event | List |
Supported feature engineering
As value
This is most basic function whereby the StreamEvent
field value is copied in to the feature map.
Example
The following example will copy the location_code
and store_id
values directly in to the feature map.
Expression based
Joule core provides the ability to deploy declarative expressions using the custom analytics processor. This has been reused within the context of feature engineering to enable users to define custom calculations within the DSL.
Example
The following example computes per event, the spend ration based utilising a Javascript expression.
Custom Plugins
Developers can extend the feature engineering capabilities by extending the AbstractFeatureEngineeringFunction
interface.
See CustomUserPlugin API documentation for further details.
Example
The following example computes per event, the scale price based utilising the MinMax
algorithm.
This example implements the AbstractFeatureEngineeringFunction
class.
Available options
Joule provides a small set of OOTB feature engineering functions.
Scripting
Define custom analytics with declarative expressions
Scaling
Normalise data with various scaling methods
Transform
Generate analytics-ready features from data
Versioning
Every feature map created is versioned using a random UUID.
The version is place directly in to the resulting map and accessed using the feature_version
key.
Last updated