INFO
Within these release notes Datameer provides information and resolutions on improvements, bug fixes, and visual changes for 11.0.x maintenance release versions.
Visit Datameer's Help Center for additional information or contact your customer service representative.
Datameer X 11.0.2
New Features
1 | Supported Hadoop Distributions: Add support for Amazon EMR 6.1.0 | DAP-40876 |
|---|---|---|
Datameer X now supports Amazon EMR version 6.1.0. |
2 | JSON Parsing: Add ability to insert a custom node into the 'JSON Tree View' | DAP-39833 |
|---|---|---|
A user can now add a custom node into the 'JSON Tree View' on a node of type 'OBJECT' or 'ARRAY' and also remove the new node. The 'Raw JSON' tab provides an edit button to edit/provide new JSON string. When in edit mode, all JSON parsing functionality is disabled. |
Improvements
1 | Import/ Export: Support for Azure Synapse (formerly Microsoft SQL Data Warehouse) | DAP-27739 |
|---|---|---|
You can now import from or export to an Azure Synapse Database. The according dialect is being supported as well. |
2 | Supported Hadoop Distributions: Add support for Amazon EMR 5.30.1 | DAP-40875 |
|---|---|---|
Datameer X now supports Amazon EMR version 5.30.1. |
3 | Supported Hadoop Distributions: Add support for Cloudera CDP-DC/ Cloudera Runtime 7.1.3 | DAP-40877 |
|---|---|---|
Datameer X now supports Cloudera CDP-DC/ Cloudera Runtime version 7.1.3. |
4 | Connection: Rewrite Google BigQuery import connector to use Google BigQuery storage API | DAP-40883 |
|---|---|---|
When importing from or exporting to Google BigQuery, the new Google BigQuery storage API is used for more efficient imports and exports.. |
5 | Authentication: Remove 'gssapi-with-mic' from Datameer's 'default.properties' | DAP-40799 |
|---|---|---|
The value 'gssapi-with-mic' is removed from the 'default.properties' key 'system.property.jsch.preferred.authentication'. The new added value is 'password,keyboard-interactive,publickey'. |
6 | Supported Hadoop Distributions: Add support for Apache Hadoop 2.7.7 and Hive 2.3.7 | DAP-40895 |
|---|---|---|
Datameer X now supports Hadoop version 2.7.7 and Hive version 2.3.7. |
7 | Workbook: Add ability to 'update' instead of 'replace' for 'Cascade Permissions' | DAP-40916 |
|---|---|---|
A user can now add permissions within 'Cascade Permissions' from a folder in the File Browser instead of fully replace the existing permissions. |
8 | Google BigQuery: Do not use temporary table when streaming from tables | DAP-40939 |
|---|---|---|
Imports from Google BigQuery now work without changes for tables, views and queries. |
9 | Plug-ins: JDBC/Snowflake - Unimplemented date/ time/ timestamp column splitting | DAP-40952 |
|---|---|---|
The plug-in 'plugin-jdbc-v2' is now adapted to the column splitting needs. |
10 | Supported Hadoop Distributions: Add support for Cloudera CDP-DC/ Cloudera Runtime 7.1.2 | DAP-40998 |
|---|---|---|
Datameer X now supports Cloudera CDP-DC/ Cloudera Runtime version 7.1.2. |
11 | Supported Hadoop Distributions: Add support for Cloudera CDP-DC/ Cloudera Runtime 7.2.0.2 | DAP-40999 |
|---|---|---|
Datameer X now supports Cloudera CDP-DC/ Cloudera Runtime version 7.2.0.2. |
Bug fixes
1 | Workbook: SQL - 'EXCEPT' and 'INTERSECT' operator null value handling | DAP-40739 |
|---|---|---|
The Datameer functions 'EXCEPT' and 'INTERSECT' work properly again. The null value handling returns an empty result as expected. |
|
Bug fixed.2 | Workbook: Validate for sheet error when changing column type which is used for an encoding sheet | DAP-40892 |
|---|---|---|
The sheet error is being registered and a proper error message is shown when changing the column type which is used for an encoding sheet. |
|
| 3 | Connection: ZooKeeper - Datameer does not close ZooKeeper connections (HBase) | DAP-40970 |
|---|---|---|
The HBase plug-in was fixed so Datameer now closes ZooKeeper connections to HBase again when the connection is saved. |
|
4 | Connection: MariaDB dialect is missing export transformation 'SqlDateTransformation' | DAP-40989 |
|---|---|---|
The needed export transformation is now added. MariaDB is changed to apply the export transformation 'SqLDateTransformation'. |
|
5 | Plug-ins: JDBC 2/ Snowflake - Integer column splitting skips table rows for import | DAP-40990 |
|---|---|---|
All splitted columns from type integer can be imported completely. |
|
6 | Workbook: Sequential triggering does not respect the execution order | DAP-41000 |
|---|---|---|
Sequential triggering executes the artifacts based on the execution order. |
|
Datameer X 11.0.1
New Features
1 | Workbook: New call to insert usage of sheets into Workbook config and details page | DAP-40722 |
|---|---|---|
A new workbook query includes the usage of a sheet. The new call includes the information and displays it in the workbook configuration and workbook details page. Therefore a user knows if a sheet must be kept or not. |
Improvements
1 | Workbook: Show each column in error state separately/ more prominent | DAP-40370 |
|---|---|---|
Columns in an error state are highlighted in an error color in the columns list inside the Workbook Sheet Inspector. Clicking the column in error state highlights the column in the sheet. Furthermore the column list height is resizable now. |
2 | Workbook: Improve filter error validation by showing why it does not validate | DAP-40594 |
|---|---|---|
When entering an invalid advanced filter definition, the error message states the problem of the filter definition. |
3 | Workbook Health Check: Provide the possibility to hide the loading screen and cancel the check | DAP-40607 |
|---|---|---|
You can now hide the Workbook Health Check loading screen and continue working while the check is executed in the background. Furthermore you can cancel the check completely. |
4 | Security: SAML authentication - role mapping proceeding | DAP-40742 |
|---|---|---|
In the 'Admin' tab in the SAML authentication you can now assign a role and/ or a group to a user directly, e.g. 'User1: ANALYST:sales'. Note that the group must exist when assigning the group to the role. |
5 | Logging: New log file to store Datameer PID (Process ID) | DAP-40761 |
|---|---|---|
A new log file named 'start-pid.log' is introduced to log the Datameer start time and the corresponding process ID. Whenever Datameer starts, a new line is being added to the log file. Find the log file under '/logs/dir'. |
6 | Workbook: Add join sheet creation validation query | DAP-40784 |
|---|---|---|
If an invalid join definition is trying to be created, a join validation provides a hint to the user why the join can not be executed. |
7 | Workbook: No consumer indication in the workbook configuration dialog if sheet is configured to be synced via Open Document Format | DAP-40818 |
|---|---|---|
The workbook configuration dialog now shows an indication that also synced sheets have a dependency consumed by. A workbook detail page contains truncated entries and a tooltip if the tables are longer than the rendered column width. |
8 | Security: Make a SAML plug-in aware of a load balancer that offloads SSL | DAP-40868 |
|---|---|---|
The load balancer offloads SSL to the Datameer SAML plug-in. A new field is provided that enables - in addition to the existing Jetty-based SSL termination - the load balancer-based SSL offloading. |
9 | Add Support for Amazon EMR 5.30.1 | DAP-40875 |
|---|---|---|
Datameer now supports Amazon EMR 5.30.1 |
10 | Security: SAML - Add option to read roles only from repository | DAP-40886 |
|---|---|---|
User roles can now be read from the repository. All SAML assertion properties (username, email, groups and roles) are configurable as well. |
Bug fixes
1 | Export: Can't export string values into MariaDB | DAP-38661 |
|---|---|---|
New Hadoop configuration parameters, that may be set in the 'Admin' tab → 'Cluster Configuration' section or during the export job configuration, override the dialect's default varchar field length. The export to MariaDB is now possible. |
|
2 | Import: Google BigQuery - Table list is empty after selecting a dataset | DAP-40542 |
|---|---|---|
The table list appears not empty any more in the import job configuration. |
|
3 | Datameer Distribution: 'Snappy-java-*.jar' is missed in installation packages for Amazon EMR 6.0.0 | DAP-40721 |
|---|---|---|
'Snappy-java-1.1.4.jar' is now bundled with the distribution. |
|
4 | Housekeeping: Set flag to DATA_DELETION_TURNED_OFF instead of DELETION_FAILED when Housekeeping data deletion is disabled | DAP-40744 |
|---|---|---|
The status column in 'filesystem_artifact_to_delete table can now have the value '2=DATA_DELETION_DISABLED'. That means that artifacts are scheduled for deletion but the distributed storage artifact deletion has been disabled via the 'housekeeping.data.deletion.disabled=true' flag. |
|
5 | Workbook: Columns.unequalCount error in join sheet | DAP-40753 |
|---|---|---|
An error is shown when submitting a join definition with duplicate keys also stating how to recover. Find more information here. |
|
6 | Workbook: IllegalArgumentException for join operation with multiple conditions | DAP-40754 |
|---|---|---|
Join operations with multiple conditions work again. |
|
| 7 | Connections: Databricks import connector throws an 'org.apache.spark.sql.AnalysisException' | DAP-40764 |
|---|---|---|
The exception is fixed and the import works as expected. |
|
8 | Datameer distribution: Show port after starting a Datameer distribution with 'bin/conductor.sh start | DAP-40765 |
|---|---|---|
The port is now displayed after starting Datameer. |
|
9 | Admin: Non-admin user can change folder ownership and select other users with the same role | DAP-40767 |
|---|---|---|
If a user has no capability 'Change Folder Owner' but the capability 'User can modify every file and folder' he is not able to change the owner of the folder. |
|
10 | Import: Columns of UUID type are being dropped from a PostgreSQL import | DAP-40794 |
|---|---|---|
The UUID type is now supported for the import and therefore the import job is executed successfully. |
|
11 | Workbook: Can't duplicate or copy a workbook if the sheet execution order does not match the view order | DAP-40805 |
|---|---|---|
Duplicating or copying a workbook works as expected and Datameer follows the sheet logical order regardless of their current sequential position. |
|
12 | Database drivers: The MariaDB database driver has a wrong name in the JDBC connection string template | DAP-40855 |
|---|---|---|
The MariaDB database JDBC connection string now has the correct string template and you can establish a connection as expected. |
|
13 | Authentication: The default LDAP user filter does not filter out objects with username or email attribute set as an empty string | DAP-40884 |
|---|---|---|
User objects with a username or email attribute set to an empty string are now filtered out and do not break the LDAP cache creation. |
|
14 | Datameer distribution: Amazon EMR should use a conform Hive version | DAP-40890 |
|---|---|---|
A Datameer distribution with Amazon EMR now uses Hive 2.3.2. |
|
15 | Datameer distribution: Amazon EMR versions should use patched Hive/ Hadoop jars if available | DAP-40891 |
|---|---|---|
Datameer uses patched Hive and Hadoop jars for compatibly reasons. |
|