Secure Impersonation on Datameer Without Super Group Requirement

Previously, secure impersonation functioned by having a single super group user, the Datameer X service user, in the Hadoop environment impersonate all authenticated users set in Kerberos. Datameer's new secure impersonation method (Native Multi User) validates each individual user separately using the user's own Kerberos keytab.  


Setup Instructions for Secure Impersonation in Native Multi User Mode

First, create a user for your Datameer X installation. This user will be called the Datameer X service user. Unless otherwise stated, perform the next steps as this user.

Preparing Datameer X for startup using the command line

Configure Datameer X in the UI

License

Install the license.

Authentication

  1. Go to the Admin tab.
  2. Click Authentication from the menu.
  3. Configure your Active Directory or LDAP (this example focus on Active Directory in the following steps) by clicking Edit .

    Important

    Enter the impersonation attribute. (E.g., userPrincipalName). This is necessary if your users are in different realms. If you don't use it then the configured default realm from your Datameer X servers Kerberos client config is taken to create the principal for the Kerberos Keytab file.

  4. Save the authentication configuration.

    Don't log out! If you log out you will not be able to login again as the current admin user. Only the superuser functionality of DM will be able to log you in again.
  5. Wait until the authentication cache has refreshed.
  6. Click Users  from the menu. Select the groups and users that should be able to authenticate.
  7. Define a new admin user. The current user is no longer valid after logout.

Hadoop cluster

  • Click on Hadoop cluster from the menu and click Edit.
  • From the Cluster Mode settings, select Multi User Kerberos Secured Hadoop Cluster.
  • Enter your name node.
  • Enter the private folder path for Datameer.
  • Select the Enable Impersonation checkbox.
  • Enter your Cluster Settings depending on your Hadoop cluster.
  • Enter under Kerberos Settings:
    • the principal for the Datameer X service user. (E.g., <name>@<realm> )
    • the path to the Datameer X service users Keytab file. 
    • the yarn principal. (E.g., yarn/_HOST@<realm>
    • the HDFS principal. (E.g., hdfs/_HOST@<realm>
    • the mapred principal. (E.g., mapred/_HOST@<realm>
  • Enter any necessary Hadoop properties for the cluster. 
  • Enter under Automatic Keytab Management Settings
    • the full path to ktutil. (E.g., /usr/bin/ktutil
    • the path to the folder where Datameer X should store the generated Keytab files. The folder needs to have the user "Datameer" as the owner and the permission 700. There is already a Keytab folder in the default installation of Datameer. (E.g., <path to your Datameer X installation>/etc/keytabs)
    • Don't set the Kerberos Realm if you already set the impersonation attribute in the Active Directory configuration . If you don't set the Kerberos Realm and you don't have the impersonation attribute set in the Active Directory configuration, the default realm from the Datameer X server Kerberos client configuration will be taken. 
    • Enter the password encryption algorithms necessary for your Kerberos installation. (E.g., aes256-cts-hmac-sha1-96, aes256-cts, arcfour-hmac
  • logout and restart Datameer

Private Folder Permissions 

Private folder permissions are automatically set up during the installation process of the secured Hadoop Distributed Filesystem. The information below is a detailed description if these settings need to be applied to other tools, such as Apache Ranger.

  • Enter the following permissions via ACL's or Ranger for common group or individual users <datameer private folder> to r-x.
  • Enter the following permissions via ACL's or Ranger for common group or individual users <datameer private folder> sub folders to rwx.

Datameer X restricts its folder to the following permissions:

folder

owner

group

user

group

others

datameer private folderdatameer service userdatameer service user grouprwx-----x
datameer private folder 1st level siblingsdatameer service userdatameer service user grouprwx----wx
individual job foldersjob ownerdefault group or grouped shared withrwx

no group sharing → ---

group allowed to edit → -wx

group allowed to view → r-x

no others sharing → ---

others allowed to edit → -wx

others allowed to view → r-x

job execution folderjob ownerdefault group or grouped shared withrwx

no group sharing → ---

group allowed to edit → --x

group allowed to view → r-x

no others sharing → ---

others allowed to edit → --x

others allowed to view → r-x

Changing ownership of a file or folder

INFO

When using secure impersonation on Datameer X in Native Multi User mode, the Datameer Service User is not a part of the Hadoop superuser group and therefore loses the privilege to change an artifact's ownership. Datameer stores data in the HDFS, and whenever you want to change the artifact's owner, the ownership of the associated files in the HDFS have to be changed as well. Changing the ownership is only possible when there are no data objects ('actual data' and 'job history') associated with this artifact in the HDFS.

Note that changing the ownership via the Set Permission option (bulk amendment) is a bug, as it does not change HDFS objects. 

To change the ownership anyway:

Note: This is not an option for artifacts whose original data sources are not available anymore.

  1. Change to the artifacts detail page. 
  2. Switch to the 'Current Data' section, click on the delete icon from the 'Operations' column and delete the actual data.
  3. Switch to the 'History' section and click on the delete icon from the 'Operations' column and delete the job history.
  4. Switch to the File Browser and click on the artifact. 
  5. From the File Browser Inspector, navigate to the section 'Owner' and change the artifacts owner to your needs. 
  6. Rerun the artifact to regenerate the data under the new ownership.