Page Comparison

Datameer X Prerequisites

...

• your Datameer X instance must be from version 7.4 or higher
• your Datameer X cluster must be run in grid mode 
• your Hadoop cluster/ HDFS must be configured to allow access to the data within the HDFS
• the plug-in 'Datameer DWH API' (including two extensions) must be installed
  NOTE: Please contact the Datameer Service to receive the INFO: The plug-in is installed as default Datameer X.
• a job must be run at least once in Datameer X and also be kept
• any data you wish to access through Spotlight must be in the 'Parquet' format (this has been the default format since Datameer 6.3)
• your user must at least have the role 'ANALYST' to access the data:
  • read-only access to the Datameer folders and files

Datameer X Running on Hadoop/ HDFS

The following prerequisites must be fulfilled from the Datameer Spotlight instanceapplies for running Datameer X on Hadoop/ HDFS:

• the Datameer X instance must be reachable:
  • the REST API endpoint URL
  • user name
  • password
• the Hadoop/ HDFS instance must be reachable:
  • the Hadoop - NameNodes
  • the Hadoop - DataNodes
• this includes being able to resolve hostnames
• all ports need to be accessible (Note that the ports might have been changed by your in-house administrators for security reasons. For specific Hadoop vendors other ports might be valid. Please view also the Hadoop vendor documentation): 
  • Hadoop NameNode client port: normally 8020, 9000 or 54310
  • Hadoop DataNodes: normally 50010 and 50020

...

• Spotlight must be able to read the data files at the S3 folder/ file locations, provided by the Datameer DWH API:
  • access key/secret must be known in the appropriate account that can read from Datameer X's internal storage bucket
  • have

    these credentials need to be

    set as connection options

    provided as additional "Connection Options" inside Spotlight when setting up the Datameer X connection using this format

    Code Block
    fs.s3a.access.key=your_access_key fs.s3a.secret.key=your_secret_key

  • these connection options only needed to be provided once for each user who needs to create connections to Datameer X. Future connections created by the same user will inherit these options.

These connection options are for creating connections only. Once a connection to Datameer X is created, other Spotlight users only need to provide their normal username and password to access that data (assuming they have permission in Datameer X to view that data).