Secure Grid Mode Configuration

Owned by Juliane Wetzel
Apr 30, 2020
3 min read

Secure Cluster Mode


Setting

Description

HDFS Namenode

Full URI to the Namenode, e.g., hdfs://machine.company.com:8020

Datameer X Private Folder

Path to the Datameer X user's directory on HDFS

Job Tracker

Hostname and port of the JobTracker e.g., machine.company.com:8020

Datameer X Principal

Fully qualified Kerberos principal for the Datameer X user, e.g., user/machine.company.com@COMPANY.COM

Datameer X Principal Keytab Path

Absolute path of the keytab file containing the credentials of the Datameer X user, e.g., /path/to/file.keytab

Yarn Principal

Fully qualified Kerberos principal for the Namenode service user, user will almost always be yarn (e.g., yarn/machine.company.com@COMPANY.COM.) This is the same property as you would configure on the NameNode (i.e., should have the NameNode's hostname.)

yarn.resourcemanager.principal parameter from the Hadoop configuration.

HDFS Principal

Fully qualified Kerberos principal for the Namenode service user, user is almost always be hdfs (e.g., hdfs/machine.company.com@COMPANY.COM). This is the same property as you would configure on the NameNode, (i.e., should have the NameNode's hostname.)

hdfs.namenode.kerberos.principal parameter from the Hadoop configuration.

Mapred Principal

Fully qualified Kerberos principal for the JobTracker service user, user is almost always be mapred (e.g., mapred/machine.company.com@COMPANY.COM) This is the same property as you would configure on the NameNode, (i.e., should have the JobTracker's hostname.)

mapreduce.jobhistory.principal parameter from the Hadoop configuration.

Creating a Kerberos keytab

To create a keytab file for a Kerberos principal, das/machine.company.com@COMPANY.COM, run the ktuil application on a machine with Kerberos client software and access to the Kerberos realm.

ktutil: addent -password -p das/machine.company.com@COMPANY.COM -k 1 -e des3-cbc-sha1
<ENTER PASSWORD>
ktutil: wkt das.keytab

This action creates a keytab file das.keytab which can now be used to authenticate the Kerberos principal.

Depending on your Kerberos realm's configured encryption types, the values for the -e argument might change. To find the avaliable encryption types look for the libdefaults section in your Kerberos configuration file, usually /etc/krb5.conf:

[libdefaults]
       default_realm = LOCAL.NETWORK
       dns_fallback = no
       noaddresses = TRUE
       default_tgs_enctypes = des3-cbc-sha1
       default_tkt_enctypes = des3-cbc-sha1
       permitted_enctypes = des3-cbc-sha1