Page Comparison

Info

title	INFO

Within these release notes Datameer X provides information and resolutions on improvements, bug fixes, and visual changes for 11.0.x maintenance release versions.

Visit Datameer's Help Center for additional information or contact your customer service representative.

Section

Datameer X X 11.0.2

New Features

1	Supported Hadoop Distributions: Add support for Amazon EMR 6.1.0	DAP-40876
	Datameer X X now supports Amazon EMR version 6.1.0.

2	JSON Parsing: Add ability to insert a custom node into the 'JSON Tree View'	DAP-39833
	A user can now add a custom node into the 'JSON Tree View' on a node of type 'OBJECT' or 'ARRAY' and also remove the new node. The 'Raw JSON' tab provides an edit button to edit/provide new JSON string. When in edit mode, all JSON parsing functionality is disabled.

Improvements

1	Import/ Export: Support for Azure Synapse (formerly Microsoft SQL Data Warehouse)	DAP-27739
	You can now import from or export to an Azure Synapse Database. The according dialect is being supported as well.

2	Supported Hadoop Distributions: Add support for Amazon EMR 5.30.1	DAP-40875
	Datameer X X now supports Amazon EMR version 5.30.1.

3	Supported Hadoop Distributions: Add support for Cloudera CDP-DC/ Cloudera Runtime 7.1.3	DAP-40877
	Datameer X X now supports Cloudera CDP-DC/ Cloudera Runtime version 7.1.3.

4	Connection: Rewrite Google BigQuery import connector to use Google BigQuery storage API	DAP-40883
	When importing from or exporting to Google BigQuery, the new Google BigQuery storage API is used for more efficient imports and exports..

5	Authentication: Remove 'gssapi-with-mic' from Datameer's 'default.properties'	DAP-40799
	The value 'gssapi-with-mic' is removed from the 'default.properties' key 'system.property.jsch.preferred.authentication'. The new added value is 'password,keyboard-interactive,publickey'.

6	Supported Hadoop Distributions: Add support for Apache Hadoop 2.7.7 and Hive 2.3.7	DAP-40895
	Datameer X X now supports Hadoop version 2.7.7 and Hive version 2.3.7.

7	Workbook: Add ability to 'update' instead of 'replace' for 'Cascade Permissions'	DAP-40916
	A user can now add permissions within 'Cascade Permissions' from a folder in the File Browser instead of fully replace the existing permissions.

8	Google BigQuery: Do not use temporary table when streaming from tables	DAP-40939
	Imports from Google BigQuery now work without changes for tables, views and queries.

9	Plug-ins: JDBC/Snowflake - Unimplemented date/ time/ timestamp column splitting	DAP-40952
	The plug-in 'plugin-jdbc-v2' is now adapted to the column splitting needs.

10	Supported Hadoop Distributions: Add support for Cloudera CDP-DC/ Cloudera Runtime 7.1.2	DAP-40998
	Datameer X X now supports Cloudera CDP-DC/ Cloudera Runtime version 7.1.2.

11	Supported Hadoop Distributions: Add support for Cloudera CDP-DC/ Cloudera Runtime 7.2.0.2	DAP-40999
	Datameer X X now supports Cloudera CDP-DC/ Cloudera Runtime version 7.2.0.2.

Bug fixes

1	Workbook: SQL - 'EXCEPT' and 'INTERSECT' operator null value handling	DAP-40739
	The Datameer X functions 'EXCEPT' and 'INTERSECT' work properly again. The null value handling returns an empty result as expected.	Bug fixed.

2	Workbook: Validate for sheet error when changing column type which is used for an encoding sheet	DAP-40892
	The sheet error is being registered and a proper error message is shown when changing the column type which is used for an encoding sheet.	Bug fixed.

3	Connection: ZooKeeper - Datameer X does not close ZooKeeper connections (HBase)	DAP-40970
	The HBase plug-in was fixed so Datameer X now closes ZooKeeper connections to HBase again when the connection is saved.	Bug fixed.

4	Connection: MariaDB dialect is missing export transformation 'SqlDateTransformation'	DAP-40989
	The needed export transformation is now added. MariaDB is changed to apply the export transformation 'SqLDateTransformation'.	Bug fixed.

5	Plug-ins: JDBC 2/ Snowflake - Integer column splitting skips table rows for import	DAP-40990
	All splitted columns from type integer can be imported completely.	Bug fixed.

6	Workbook: Sequential triggering does not respect the execution order	DAP-41000
	Sequential triggering executes the artifacts based on the execution order.	Bug fixed.

Section

Datameer X X 11.0.1

New Features

1	Workbook: New call to insert usage of sheets into Workbook config and details page	DAP-40722
	A new workbook query includes the usage of a sheet. The new call includes the information and displays it in the workbook configuration and workbook details page. Therefore a user knows if a sheet must be kept or not.

Improvements

1	Workbook: Show each column in error state separately/ more prominent	DAP-40370
	Columns in an error state are highlighted in an error color in the columns list inside the Workbook Sheet Inspector. Clicking the column in error state highlights the column in the sheet. Furthermore the column list height is resizable now.

2	Workbook: Improve filter error validation by showing why it does not validate	DAP-40594
	When entering an invalid advanced filter definition, the error message states the problem of the filter definition.

3	Workbook Health Check: Provide the possibility to hide the loading screen and cancel the check	DAP-40607
	You can now hide the Workbook Health Check loading screen and continue working while the check is executed in the background. Furthermore you can cancel the check completely.

4	Security: SAML authentication - role mapping proceeding	DAP-40742
	In the 'Admin' tab in the SAML authentication you can now assign a role and/ or a group to a user directly, e.g. 'User1: ANALYST:sales'. Note that the group must exist when assigning the group to the role.

5

Logging: New log file to store Datameer X PID (Process ID)

DAP-40761

A new log file named 'start-pid.log' is introduced to log the Datameer X start time and the corresponding process ID. Whenever Datameer X starts, a new line is being added to the log file. Find the log file under '/logs/dir'.

6	Workbook: Add join sheet creation validation query	DAP-40784
	If an invalid join definition is trying to be created, a join validation provides a hint to the user why the join can not be executed.

7	Workbook: No consumer indication in the workbook configuration dialog if sheet is configured to be synced via Open Document Format	DAP-40818
	The workbook configuration dialog now shows an indication that also synced sheets have a dependency consumed by. A workbook detail page contains truncated entries and a tooltip if the tables are longer than the rendered column width.

8	Security: Make a SAML plug-in aware of a load balancer that offloads SSL	DAP-40868
	The load balancer offloads SSL to the Datameer X SAML plug-in. A new field is provided that enables - in addition to the existing Jetty-based SSL termination - the load balancer-based SSL offloading.

9	Add Support for Amazon EMR 5.30.1	DAP-40875
	Datameer X now supports Amazon EMR 5.30.1

10	Security: SAML - Add option to read roles only from repository	DAP-40886
	User roles can now be read from the repository. All SAML assertion properties (username, email, groups and roles) are configurable as well.

Bug fixes

1	Export: Can't export string values into MariaDB	DAP-38661
	New Hadoop configuration parameters, that may be set in the 'Admin' tab → 'Cluster Configuration' section or during the export job configuration, override the dialect's default varchar field length. The export to MariaDB is now possible.	Bug fixed.

2	Import: Google BigQuery - Table list is empty after selecting a dataset	DAP-40542
	The table list appears not empty any more in the import job configuration.	Bug fixed.

3	Datameer X Distribution: 'Snappy-java-*.jar' is missed in installation packages for Amazon EMR 6.0.0	DAP-40721
	'Snappy-java-1.1.4.jar' is now bundled with the distribution.	Bug fixed.

4	Housekeeping: Set flag to DATA_DELETION_TURNED_OFF instead of DELETION_FAILED when Housekeeping data deletion is disabled	DAP-40744
	The status column in 'filesystem_artifact_to_delete table can now have the value '2=DATA_DELETION_DISABLED'. That means that artifacts are scheduled for deletion but the distributed storage artifact deletion has been disabled via the 'housekeeping.data.deletion.disabled=true' flag.	Bug fixed.

5	Workbook: Columns.unequalCount error in join sheet	DAP-40753
	An error is shown when submitting a join definition with duplicate keys also stating how to recover. Find more information here.	Bug fixed.

6	Workbook: IllegalArgumentException for join operation with multiple conditions	DAP-40754
	Join operations with multiple conditions work again.	Bug fixed.

7	Connections: Databricks import connector throws an 'org.apache.spark.sql.AnalysisException'	DAP-40764
	The exception is fixed and the import works as expected.	Bug fixed.

8	Datameer X distribution: Show port after starting a Datameer X distribution with 'bin/conductor.sh start	DAP-40765
	The port is now displayed after starting Datameer.	Bug fixed.

9	Admin: Non-admin user can change folder ownership and select other users with the same role	DAP-40767
	If a user has no capability 'Change Folder Owner' but the capability 'User can modify every file and folder' he is not able to change the owner of the folder.	Bug fixed.

10	Import: Columns of UUID type are being dropped from a PostgreSQL import	DAP-40794
	The UUID type is now supported for the import and therefore the import job is executed successfully.	Bug fixed.

11	Workbook: Can't duplicate or copy a workbook if the sheet execution order does not match the view order	DAP-40805
	Duplicating or copying a workbook works as expected and Datameer X follows the sheet logical order regardless of their current sequential position.	Bug fixed.

12	Database drivers: The MariaDB database driver has a wrong name in the JDBC connection string template	DAP-40855
	The MariaDB database JDBC connection string now has the correct string template and you can establish a connection as expected.	Bug fixed.

13	Authentication: The default LDAP user filter does not filter out objects with username or email attribute set as an empty string	DAP-40884
	User objects with a username or email attribute set to an empty string are now filtered out and do not break the LDAP cache creation.	Bug fixed.

14	Datameer X distribution: Amazon EMR should use a conform Hive version	DAP-40890
	A Datameer X distribution with Amazon EMR now uses Hive 2.3.2.	Bug fixed.

15	Datameer X distribution: Amazon EMR versions should use patched Hive/ Hadoop jars if available	DAP-40891
	Datameer X uses patched Hive and Hadoop jars for compatibly reasons.	Bug fixed.

Versions Compared

Old Version 2

New Version 3

Key

Datameer X X 11.0.2

New Features

Improvements

Bug fixes

Datameer X X 11.0.1

New Features

Improvements

Bug fixes