Connectors API

Connectors are key to enable Joule to perform use cases. Given the importance of Connectors, an API has been provided to enable developers to build and extend the capabilities of the platform.

Requirements

See Setting up the environment documentation

Note all gradle commands must be execute at the root of the project directory

Development steps

1. Create project using the template

2. Define connector specification

3. Implement custom transport

4. Build, test and package

5. Deploy

Explaining each step

Step 1: Create project using the template

We have provided a project template project to quick start development. The project can be found here. Clone the template project and copy relevant code and structure to your own project

git clone [e-mail]:joule-platform/fractalworks-project-templates.git

Joule uses Gradle to manage Java dependencies. To add dependencies for your connector, manage them in the build.gradle file inside your connector's directory.

Step 2: Define connector specification

For components to be defined using the Joule DSL a specification class is required. Joule provides a AbstractTransportSpecification class that is to be extended for your implementation. This class provides core attributes that support a number of features such as:

• Batching

• Formatting

• Parsing Threads

Publisher Example

@JsonRootName(value = "templatePublisher")
public class TemplatePublisherSpecification extends AbstractTransportSpecification {

    private String someField;

    /**
     * Default and required
     */
    public TemplatePublisherSpecification() {
        super();
    }

    /**
     * Default and required
     */
    public TemplatePublisherSpecification(String name) {
        super(name);
    }
    
    @JsonProperty(value = "some field", required = true)
    public void setSomeField(String someField) {
        this.someField = someField;
    }

    @Override
    public void validate() throws InvalidSpecificationException {
        super.validate();
        // TODO: Add validation logic based upon required fields
        if (someField == null || someField.isEmpty()) {
            throw new InvalidSpecificationException("someField must be provided.");
        }
    }
    
    @JsonIgnore
    @Override
    public Class<? extends Transport> getComponentClass() {
        // TODO: Change to your transport class
        return TemplatePublisherTransport.class;
    }
}

There is an option to create a specification builder in the event of programmactically. See the TemplatePublisherSpecificationBuilder example class for a reference implementation.

For Joule to load and initialised the component the specifications need to be defined within the plugins.properties file under the META-INF/services directory

Example

# Change and add lines for your specification classes
com.fractalworks.streams.examples.transport.TemplatePublisherSpecification

Step 3: Implement custom transport

Now we can move on to building the transport class that will either consume or publish events.

Publisher Example

Publisher has two key methods to implement; initialise and publish.

// Some code
public class TemplatePublisherTransport extends AbstractPublisherTransport {

    private String someField;

    public TemplatePublisherTransport() {
        super();
        logger = LoggerFactory.getLogger(TemplatePublisherTransport.class);
    }

    public TemplatePublisherTransport(TemplatePublisherSpecification specification) {
        super(specification);
        logger = LoggerFactory.getLogger(TemplatePublisherTransport.class);
        someField = specification.getSomeField();
    }

    @Override
    public void initialize() {
        super.initialize();
        // TODO: Add custom transport initialisation code
    }

    @Override
    public void publish(Collection<StreamEvent> events) {
        // TODO: Add custom transport logic to transmit events
    }
}

Consumer Example

Consumer has two key methods to implement; initialise and start.

public class TemplateConsumerTransport extends AbstractConsumerTransport {

    public TemplateConsumerTransport() {
        super();
        logger = LoggerFactory.getLogger(TemplateConsumerTransport.class);
    }

    public TemplateConsumerTransport(TemplateConsumerSpecification specification) {
        super(specification);
        logger = LoggerFactory.getLogger(TemplateConsumerTransport.class);
    }

    @Override
    public void initialize() {
        super.initialize();
        // TODO: Add custom transport initialisation code
    }
    

    @Override
    public void start() {
        super.start();
        // TODO: Add custom transport consumer code
    }
}

Step 4: Build, test and package

The template project provides basic JUnit test to validate DSL. The project will execute these tests during the gradle build cycle and deploy to your local maven repository.

gradle build publishToMavenLocal

Step 5: Deploy

Once your package has been successfully created you are ready to deploy to a Joule project. The resulting jar artefact needs to be placed in to the userlibs directory in your Joule projects directory. See provided examples documentation for further directions.

cp build/libs/<your-connector>.jar <location>/userlibs