Page Comparison

...

Introduction

Datameer can be customized by using the Datameer SDK and public API to create plug-ins. Types of plug-ins currently available include functions, file types, export formats, and connections. Additional plug-in types will are on the roadmap to be supported in the future.

Table of Contents

Prerequisites

Java 7 JDK
Ant 1.7 or newer
Cygwin DLL (for developers using Windows)
If you are installing a plug-in that uses extension points, make sure to install the plug-in that supplies those extension points first.

Extension Points

...

Extension points

Extension point are the sockets where plug-in developers can plug-in their extensions. Currently there are a number of

Available extension points available:

Extension Point	Description
datameer.dap.sdk.importjob.ImportFileType	Import file type (e.g. , Apache log, CSV, etc.). This should be extended for adding support for custom file types.
datameer.dap.sdk.function.FunctionType	Function. This should be extended to add functions.
datameer.dap.sdk.datastore.DataStoreType	Connection type. This should be extended when support for new non-file based imports should be added. (e.g., Examples for this include Hive, HBase, etc.)
datameer.dap.sdk.authentication.AuthenticatorExtension	Defines custom authenticators.
datameer.dap.sdk.event.DatameerEventBusExtension	This should be extended to receive events provided by the Datameer Event Bus.
datameer.dap.sdk.exportjob.ExportFileType	Export file type (e.g., CSV, Avro, Parquet, etc...). This should be extended to add support for custom export file types.

Extensions

Extensions can be added for the available extension points. Every plug-in can contribute an arbitrary number of extensions to the Datameer system.

...

Plug-ins can be developed using the plug-in SDK that is included in the Datameer release. When you have installed Datameer you will find the plug-in SDK in the following location where <version> is the version of Datameer you are using:

No Format
plugin-sdk-<version>.zip

Installing the Datameer SDK

To install the SDK, copy the zip file to your development machine and install it to the location of your choice using the following commands:

...

You are now ready to develop your first plug-in.

...

Creating a

...

plug-in

To create a plug-in:

Run the provided script to create a new plug-in project:
No Format
$DAS_PLUGIN_SDK_HOME/bin/create-das-plugin.sh <project target dir>
For example:
No Format
$DAS_PLUGIN_SDK_HOME/bin/create-das-plugin.sh ~/development/das/my-first-plugin
This creates a plug-in project under under ~/development/das/my-first-plugin
Change directories to the newly created project:
No Format
cd ~/development/das/my-first-plugin
Create Eclipse project files:
No Format
ant./gradlew cleanEclipse eclipse
Import the plug-in project to Eclipse using File -> Import ... -> Existing Projects into Workspace > -> select the project directory.dir
You are now ready to start developing Datameer extensions.

...

In the plugin.xml, add the name of the extension point you want to add.

For example:

Code Block

<?xml version="1.0" encoding="UTF-8"?>
<plugin
   id="plugin-example"
   name="Example Plugin with Extension Point"
   version="${datameer.version}"
   provider-name="Datameer, Inc.">

   <requires>
      <import plugin="das.sdk"/>
   </requires>

   <extension-point id="datameer.plugin.example.ExampleExtensionPoint" name="Example Extension Point" />
</plugin>

Open your extension point and define your methods.
For example:

Code Block

package datameer.plugin.example;

import datameer.com.google.common.annotations.Beta;
import datameer.dap.sdk.plugin.SingletonExtension;

public abstract class ExampleExtensionPoint implements SingletonExtension {

    public static final String EXTENSION_POINT_NAME = ExampleExtensionPoint.class.getName();

    @Override
    public final String getExtensionPoint() {
        return EXTENSION_POINT_NAME;
    }
  
    /**
   Add your extension point specific methods here
   **/
    public abstract Sting foo();

}

Documentation

The Javadoc of the Datameer SDK is available . from:

No Format
$DAS_PLUGIN_SDK_HOME/doc/index.html

Writing

...

extensions

See the following tutorials:

Plug-in File Format

Datameer plug-ins are zip files of the following structure:

File / Folder	Description
`<root-dir>/lib/compile/`	All jar files in this directory will be added to the plug-in classpath.
`<root-dir>/lib/provided/`	Only used for developing the plug-in. Jar files under this directory will be added to the classpath of your IDE. Ignored by Datameer.
`<root-dir>/lib/runtime/`	Only used for developing the plug-in. Jar files under this directory will be added to the classpath of your IDE. Ignored by Datameer.
`<root-dir>/lib/test/`	Only used for developing the plug-in. Jar files under this directory will be added to the classpath during unit test execution. Ignored by Datameerdependencies necessary to run your plugin.
`<root-dir>/classes-main/`	This folder will be added to the plug-in classpath. Datameer will scan all classes for classes for annotations or implemented interfaces that are relevant to Datameer plug-in development.
`<root-dir>/plugin.xml`	Plug-in manifest containing plug-in meta data.

...

metadata. It defines also what dependencies are added to the classpath of your plugin.

Versioning

Edit the files plugin.xml and file build.properties to gradle to assign the appropriate names name, description, and version numbers number before the plug-in is built.

Datameer will show shows the name in the UI which is used in the name tag.

Code Block

language	xml
title	<path/workspace/plugin-name>/plugin.xml.template

<plugin
   id="plugin-name"
   name="Plugin Name"
   version="<SDK-version>-<plugin-version>"
   provider-name="MyCompany">
...

The plug-in file name is built from the name and version tag.

Code Block

language	xml
title	<path/workspace/plugin-name>/build.properties

sdk.home=<path-to-SDK>
version=<SDK-version>-<plugin-version>
...

Note

Keep the following rules in mind when filling in the ID and name of a custom plug-in:

1. The values of the ID and the name must be different. If not, Datameer shows an empty plug-in list
2. The value of the ID can't contain space characters.

Installing and Testing

...

Plug-

...

ins

Info
Learn more about installing a plug-in though the user interface (UI)about /wiki/spaces/DAS60/pages/4620161268.

Bundle your plug-in as zip file:
No Format
ant zip ./gradlew createPlugin
In Datameer, go to the the Admin Tab > > Plug-ins page page.
At the bottom of the page under the Upload Plug-ins heading, click Choose File.
Select the file and click click Upload.

...

title	Pausing jobs being submitted to the cluster

...

Version	Old Version 1	New Version 2
Changes made by	Charles Bishop (Deactivated)	Charles Bishop (Deactivated)
Saved on	Jan 30, 2018	Apr 23, 2018

Versions Compared

Key

Introduction

Prerequisites

Extension Points

Extension points

Extensions

Installing the Datameer SDK

Creating a

plug-in

Documentation

Writing

extensions

Plug-in File Format

Versioning

Installing and Testing

Plug-

ins