Using the Plug-in SDK

Introduction

Datameer X can be customized by using the Datameer X SDK and public API to create plug-ins. Types of plug-ins currently available include functions, file types, export formats, and connections. Additional plug-in types are on the roadmap to be supported in the future.

Prerequisites

Java 8 JDK

Extension points

Extension point are the sockets where plug-in developers can plug-in their extensions.

Available extension points:

Extension Point	Description
datameer.dap.sdk.importjob.ImportFileType	Import file type (e.g. Apache log, CSV, etc.). This should be extended for adding support for custom file types.
datameer.dap.sdk.function.FunctionType	Function. This should be extended to add functions.
datameer.dap.sdk.datastore.DataStoreType	Connection type. This should be extended when support for new non-file based imports should be added. Examples for this include Hive, HBase, etc.
datameer.dap.sdk.authentication.AuthenticatorExtension	Defines custom authenticators.
datameer.dap.sdk.event.DatameerEventBusExtension	This should be extended to receive events provided by the Datameer X Event Bus.
datameer.dap.sdk.exportjob.ExportFileType	Export file type (e.g.CSV, Avro, Parquet, ...). This should be extended to add support for custom export file types.

Extensions

Extensions can be added for the available extension points. Every plug-in can contribute an arbitrary number of extensions to the Datameer X system.

Development

Plug-ins can be developed using the plug-in SDK that is included in the Datameer X release. When you have installed Datameer X you find the plug-in SDK in the following location where <version> is the version of Datameer X you are using:

plugin-sdk-<version>.zip

Installing the Datameer X SDK

To install the SDK, copy the zip file to your development machine and install it to the location of your choice using the following commands:

cd <destination>
unzip /<path>/plugin-sdk-<version>.zip
export DAS_PLUGIN_SDK_HOME=<path>

You are now ready to develop your first plug-in.

Creating a plug-in

To create a plug-in:

Run the provided script to create a new plug-in project:

$DAS_PLUGIN_SDK_HOME/bin/create-das-plugin.sh <project target dir>

For example:

$DAS_PLUGIN_SDK_HOME/bin/create-das-plugin.sh ~/development/das/my-first-plugin

This creates a plug-in project under ~/development/das/my-first-plugin

Change directories to the newly created project:
```
cd ~/development/das/my-first-plugin
```
Create Eclipse project files:
```
./gradlew cleanEclipse eclipse
```
Import the plug-in project to Eclipse using File -> Import ... -> Existing Projects into Workspace -> select the project dir
You are now ready to start developing Datameer X extensions.

Documentation

The Javadoc of the Datameer X SDK is available from:

$DAS_PLUGIN_SDK_HOME/doc/index.html

Writing extensions

See the following tutorials:

Plug-in File Format

Datameer X plug-ins are zip files of the following structure:

File / Folder	Description
`<root-dir>/lib/compile/`	All dependencies necessary to run your plugin.
`<root-dir>/classes/`	This folder will be added to the plug-in classpath. Datameer X will scan all classes for classes for annotations or implemented interfaces that are relevant to Datameer X plug-in development.
`<root-dir>/plugin.xml`	Plug-in manifest containing plug-in metadata. It defines also what dependencies are added to the classpath of your plugin.

Versioning

Edit the file build.gradle to assign the appropriate name, description, and version number before the plug-in is built.

Datameer X shows the name in the UI which is used in the name tag.

<path/workspace/plugin-name>/plugin.xml.template

<plugin
   id="plugin-name"
   name="Plugin Name"
   version="<SDK-version>-<plugin-version>"
   provider-name="MyCompany">
...

Keep the following rules in mind when filling in the ID and name of a custom plug-in:

1. The values of the ID and the name must be different. If not, Datameer X shows an empty plug-in list
2. The value of the ID can't contain space characters.

Installing and Testing Plug-ins

Learn more about installing a plug-in through the user interface (UI).

Bundle your plug-in as zip file:
```
./gradlew createPlugin
```
In Datameer X go to the Admin Tab > Plug-ins page.
At the bottom of the page under the Upload Plug-ins heading, click Choose File.
Select the file and click Upload.