Active Directory and LDAP

INFO

The LDAP and Active Directory authenticators available in Datameer X provide remote authentication services for Datameer X users. Administrators can configure Datameer X to use their existing LDAP or Active Directory systems as the authenticator of record allowing for centralized management of user accounts and credentials outside of Datameer X. Users can authenticate with Datameer X using existing credentials which are verified against the remote system on every login. If the remote system no longer sanctions the user, access to Datameer X is denied. This simplifies Datameer X administration and allows end-users to use familiar single sign-on credentials when accessing Datameer.

TIP

For accessing your LDAP service over SSL, see Configuring Secure LDAP (LDAPS).

Users and Groups

Users

INFO

When using the LDAP Authenticator, credentials are authenticated at the time of login directly against the remote service. In order for Datameer X to manage authorization and object permissions, a Datameer X user entity is required to represent the remote user. It useful to think of users being imported into Datameer X from LDAP with the remote server always ensuring that users have valid, active credentials.

When a user is imported into Datameer X, a user entity is created using the remote unique identifier as the Datameer X username and populated with other account details, i.e. the email address.

INFO

If authenticating with Active Directory or LDAP, users executing jobs must be listed in the authenticator. For added security, executing jobs as a users not present in the authenticator is not possible.

Groups

INFO

During import, Datameer X creates groups, based on the groups contained in the LDAP directory. This allows the use of existing LDAP groups for Datameer X access control as the group memberships are mirrored across systems. The users' group memberships and user details are updated on every login to ensure that any changes to authorization policies in LDAP are reflected in every Datameer X session. Since we rely on LDAP groups when the LDAP Authenticator is in use, there is no facility to create groups within Datameer. Datameer X can handle at maximum 500 LDAP groups.

Transforming user names into lowercase characters

Many times an all lowercase username is required for proper Unix group authentication. Instead of needing to change all user names to lowercase characters, Datameer X has the following property file to transform a username into all lowercase before passing it to the hadoop client API(s) to account for this issue:

hadoop.security.auth_to_local=RULE:[1:$1@$0](.*@EC2.INTERNAL)s/(.*)@EC2.INTERNAL/$1/L RULE:[2:$1@$0](.*@EC2.INTERNAL)s/(.*)@EC2.INTERNAL/$1/L DEFAULT

To enable this feature, add the above property file to the custom Hadoop properties.

Apache hadoop.security.auth_to_local documentation.

Preparing the Authentication

Warning

Make sure you have the superuser mode enabled in the 'live.properties' and have the password handy when configuring the authentication. For that, check if the deployment property 'das.superuser.enabled' is true and have the values for 'das.superuser.username' and 'das.superuser.password' handy.

NOTE: If you are using a custom deploy mode via 'DAS_DEPLOY_MODE' then use the properties file for that mode, live.properties is the default).

# You can set a super user account here that has the ADMIN role. This can be used to login even if
# the external authentication service doesn't work
das.superuser.enabled=true
das.superuser.username=superuser
das.superuser.password=password

Configuring an Active Directory and LDAP Authentication

To configure Datameer X to use your LDAP or Active Directory service:

Click on the "Admin" tab and select "Authentication". The page 'Authentication' opens.
Click "Edit". The authentication configuration page opens.
Select "Remote Authentication System", select "ActiveDirectory/ LDAP" from the drop-down and click "Next".
Enter the username and password for the default user.
Enter the server URL.
INFO: The URL should have the form of 'ldap://server:port'. The port numbers are: LDAP/S - 389/636 or Active Directory - 3268.
If needed, select "Use default user" if the server connection should use the default user to connect.
If needed, enter the username and password.
Enter the search base query.
Enter the user definition query.
If needed, enter the group definition query.
If needed, define the pagination control.
If you want to add multiple LDAP server, click "Add server connection". Another 'Server Connection' section opens and can be configured and deleted if non needed furthermore.
Enter the username attribute that defines a username and must be unique across all users.
Enter the email attribute.
If needed, enter the attributes for impersonation and/ or salt.
Enter the attribute for group names.
If needed, enter further group related attributes as well as group relates fields, e.g. 'Virtual Group'.
INFO: The checkbox 'Mixed mode' is enabled by default and enables both internal user management as well as SAML. Having this disabled, the internal user management is disabled.
If needed, enter group filters to include or exclude groups with regex.
If needed, check the checkbox "Ignore Case" to perform a case sensitive match when filtering group names.
Confirm the configuration with "Save". Configuring Datameer X to use LDAP or Active Directory authentication is finished.

Importing Users/Groups

Once you have successfully configured your LDAP connection, you can now import users into Datameer X enabling users to access the system. Because AD/LDAP is active, Datameer's internal authentication method is disabled. Therefore, admins must create all groups needed in the AD and add the users there before importing them. To import, click Users from the Admin menu.

From the Add Users and Groups from External Authenticator a list is populated with users/groups from the LDAP server. The search bar looks for a string contained in the User Name, email, or Group Name.

Select the user/group, assign a role, and click Add. Each user must have a role. Once a user/group has been added, the name is greyed out from the list. That user/group has access to Datameer X once the cache has been refreshed.

In order to have access to Group Authentication tab, the Datameer X license must support unlimited users.

Users/Groups that have been added from the LDAP server is displayed and can be edited.

Highlight multiple users/groups to to perform bulk role updates or deletions. It is also possible to bulk import users from your LDAP authenticator.

User Details Read-Only

The username and email address aren't editable as they are provided bythe LDAP service. Only Enable/Disable, Roles, and comments are editable from this screen.

Advanced Topics

Cache settings

Datameer X loads the entire set of LDAP users into a cache that is periodically refreshed from LDAP. This offers tremendous performance benefits and enables features like search and browse in the UI. By default, Datameer X refreshes this cache every hour. This means that modification to LDAP data might not be reflected until the cache is refreshed.

This doesn't impact authentication credentials or removal of users from LDAP. Credential checks are done against the live service, not cached data. Also user data populated at login reflects up-to-date information. The cached data only impacts the importing of users and listing of available users

To change the cache refresh settings you can set the system property ldap.authenticator.available.users.refresh.interval.minutes to whatever minute value you desire. For most installations, the 60 minute setting makes sense.

Enabling and disabling remote authentication

While setting up the system or when doing maintenance, you might find it necessary to switch off the LDAP Authenticator. When the LDAP Authenticator is disabled, all user's which were imported from LDAP are automatically disabled and the cache is dropped. When the authenticator is re-enabled, all users are then re-enabled and ready for use. You don't have to re-import any users. Also note that the cache has been re-built.

Skipping users

Datameer X builds a list of all available users for import based on the authenticator configuration. During this process, Datameer X dropS users from this list if:

The user doesn't have a username
The user doesn't have an email address
The user isn't a member of a group (see Using a Default LDAP Group for a workaround)

If you don't see users that you are expecting available for import to Datameer X check your logs and look for WARN level message such as:

LdapUser has empty email address, will be filtered from available users :
LdapUser has empty username, will be filtered out from available users :
LdapUser has no groups and default group is disabled, will be filtered from available users :

Seeing these types of messages often signifies a misconfiguration.

Using a default LDAP group

By default when constructing the list of users available for import, Datameer X skips any users that aren't members of a group in LDAP. This behavior is desirable since this is most likely a mis-configuration and Datameer X requires user to belong to a group. However, it is possible that your LDAP users haven't been assigned to a group or that for some reason the Datameer X authenticator can't resolve any. In this case, you can set a system property - ldap.authenticator.use.default.group=true - creating a default group for imported LDAP users. The group LDAP_USERS contains every user not assigned to another group, be aware of this when setting up group permissions.

Limiting LDAP user results

Many LDAP providers, Active Directory included, limit the number of results returned in most searches. When this limit it is reached, no more results are sent to the client in a single request. This is problematic, especially if you don't have the ability to configure the LDAP service to allow more results. If your LDAP installation is sufficiently large and your service is configured to limit results in this way, Datameer X might fail to load any users at all. To get around this issue, Datameer X offers several configuration options:

Use a more restrictive search base

In the Query Options section of the Authenticator configuration screen, you can define a search base that limits the scope of the LDAP query. For example, if we have a search base for a Datameer X LDAP server DC=datameer,DC=local and it contains 3,000 users which is over the limit of 1,000 results, we can refine it to include only the organizational unit we want, OU=BarUnit,DC=Datameer X DC=local which contains only 1,000 users.

Now we can successfully import users from the organizational unit BarUnit and below in the LDAP tree.

Add constraints to the user definition

The Query Options section also contains a field for configuring a User Filter. This is analogous to an SQL WHERE clause used when Datameer X queries for LDAP users. You can filter any attribute associated with a user in your LDAP server. As an example, we could filter users based on their membership in either the 'Finance' or 'IT' department. This would make only users matching the filter available for import into Datameer. The filter is defined using normal LDAP query syntax:

"LDAP Queries"

The above query says, "Find objects which are of objectClass 'person' and have a department value of either 'Finance' or 'IT'". For more details about LDAP query syntax, go here.

Set a custom MaxResults setting for the Datameer X LDAP user

If the Datameer X LDAP needs to list large numbers of users, a user specific setting can be applied in most LDAP providers. This would allow this specific user to exceed the default query result settings for this domain. In Active Directory, this is achieved by setting a MaxResults value for the Datameer X LDAP user.