Deserializers

Joule ships with the ability to parse domain types using a custom deserialiser along with standard StreamEvent deserialisers

Deserialization Specification

The DeserializerSpecification class is used to defined how StreamEvents are serialised for downstream consumption.

Attributes

Attribute	Description	Type
parser	Parser interface to convert a Object to a collection of stream events	StreamEventParser Default: StreamEventJsonDeserializer
compressed	If passed payload is compressed	Boolean Default: false
batch	Is payload a batch os events	Boolean Default: false
properties	Specific properties required for custom deserialization process	Properties

StreamEventParser

Parser interface to converts an Object to a collection of stream events.Developers are expected to provide domain specific implementations if there is a need to publish non-Joule events.

/**
* Parse passed byte array to collection of {@link StreamEvent}s
*
* @param payload byte array to translate
* @return collection of stream events
* @throws TranslationException is thrown when a translation failure occurs
*/
Collection<StreamEvent> translate(Object payload) throws TranslationException;

Available Implementations

Format	DSL	Class
json	json deserializer	StreamEventJsonDeserializer
csv	csv deserializer	StreamEventCSVDeserializer
object	object deserializer	StreamEventObjectDeserializer
avro	avro deserializer	AvroDomainDeserializer

Example

For the provided implementations a full package namespace is required. For the above implementations use com.fractalworks.streams.sdk.codec

deserializer:
   parser: 
      json deserializer: {} 

AVRO

This deserializer has extra setting to support ability to load the target schema. Currently only local schema files are supported with schema registry support on request.

Attribute	Description	Type	Required
schema	Path and name of schema file	String
field mapping	Map source feilds to map to a target StreamEvent field	Map<String,String>

Example

deserializer:
   parser: 
      avro deserializer: 
         schema: /home/myapp/schema/customer.avro

CSV

This parser has extra optional setting to support ability to custom parsing through the use of defining a custom header.

Attribute	Description	Type	Required
type map	Map of fields to types. See supported types	Map<String,Type>
delimiter	Define a custom file delimiter	String Default: ","
date format	Provide a custom date format.	String Default: yyyy/MM/dd

Example

deserializer:
    parser: 
        csv deserializer:
            type map:
               symbol: STRING
               rate:   DOUBLE

Supported Types

Following types are supported

• DOUBLE, FLOAT, LONG, INTEGER, SHORT, BYTE, BOOLEAN, STRING, DATE

Custom Example

@JsonRootName(value = "quote parser")
public class QuoteToStreamEventParser implements StreamEventParser {

    public QuoteToStreamEventParser() {
        // Required
    }

    @Override
    public Collection<StreamEvent> translate(Object o) throws TranslationException {
        Collection<StreamEvent> events = null;
        if(o instanceof Quote){
            Quote quote = (Quote) o;
            StreamEvent event = new StreamEvent("quote");
            event.setEventTime(quote.time());
            event.addValue("symbol", quote.symbol());
            event.addValue("bid", quote.bid());
            event.addValue("ask", quote.ask());
            event.addValue("volatility", quote.volatility());
            event.addValue("volume", quote.volume());
            event.addValue("date", quote.date());

            events = Collections.singletonList( event);
        }
        return events;
    }
}