Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

...

...

...

...

...

Table of Contents

Prerequisites

Info
titleINFO

Minimum access level required for importing data from an S3 bucket to Datameer.

The Account/ IAM Role one uses in the S3 connection should be allowed to perform the following actions:

  • GetObject

  • GetObjectAttributes

  • ListBucket (should be applied for an entire bucket but could be restricted to a specific directory(s) via the Condition key - "Condition": { "StringLike": {"s3:prefix": "<path>/*" }} )

For an edge case suggestion, please check the community article Bucket policy for S3 Import in multi-tenant Datameer instance.

Usable Custom Properties

Disabling the IAM Role Authentication Option

...

  1. Click the + (plus) button and select Connection or right-click in the browser and select Create new > Connection.
  2. From drop-down list, select S3  as the connection type. Click Next.
  3. Add your S3 bucket name and select the authentication mode: 
    'Access Key and Secret Code pair' or 'IAM role' or 'IAM assumed role'. Datameer X supports import/export from/to S3 encrypted buckets. 
    INFO: Selecting 'IAM role' will use the IAM role associated with the EC2 instance or the EMR cluster. Selecting the  'IAM assumed role' will use the IAM role associated with the EC2 instance or the EMR cluster to assume a specific role.

    Add the root path prefix, if necessary.

    Indicate if the connection should be used for import, export, or both.

    For the encryption support, select between Amazon Web Services, AES256 and KMS.

  4. Click Next.
  5. If required, add a description and click Save.

Anchor
import
import
Importing Data with a S3 Connector

Note

This connector isn't able to import from S3 buckets without access to read the metadata over the  getObjectMetadata() method.

After configuring a connection to S3, you can set up an import job to access the connection.

...