Versions Compared

Old Version 1

changes.mady.by.user Juliane Wetzel

Saved on

compared with

New Version 2

changes.mady.by.user Juliane Wetzel

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Info
titleINFO

Within these release notes Datameer provides information and resolutions on improvements, bug fixes, and visual changes for 11.0.x maintenance release versions.  

Visit Datameer's Help Center for additional information or contact your customer service representative. 

Section

Datameer X 11.0.2

New Features

1

Supported Hadoop Distributions: Add support for Amazon EMR 6.1.0DAP-40876

Datameer X now supports Amazon EMR version 6.1.0.


2

JSON Parsing: Add ability to insert a custom node into the 'JSON Tree View'DAP-39833

A user can now add a custom node into the 'JSON Tree View' on a node of type 'OBJECT' or 'ARRAY' and also remove the new node. The 'Raw JSON' tab provides an edit button to edit/provide new JSON string. When in edit mode, all JSON parsing functionality is disabled. 



Improvements

1

Import/ Export: Support for Azure Synapse (formerly Microsoft SQL Data Warehouse)DAP-27739

You can now import from or export to an Azure Synapse Database. The according dialect is being supported as well.


2

Supported Hadoop Distributions: Add support for Amazon EMR 5.30.1DAP-40875

Datameer X now supports Amazon EMR version 5.30.1.


3

Supported Hadoop Distributions: Add support for Cloudera CDP-DC/ Cloudera Runtime 7.1.3DAP-40877

Datameer X now supports Cloudera CDP-DC/ Cloudera Runtime version 7.1.3.


4

Connection: Rewrite Google BigQuery import connector to use Google BigQuery storage APIDAP-40883

When importing from or exporting to Google BigQuery, the new Google BigQuery storage API is used for more efficient imports and exports..


5

Authentication: Remove 'gssapi-with-mic' from Datameer's 'default.properties'DAP-40799

The value 'gssapi-with-mic' is removed from the 'default.properties' key 'system.property.jsch.preferred.authentication'. The new added value is 'password,keyboard-interactive,publickey'.


6

Supported Hadoop Distributions: Add support for Apache Hadoop 2.7.7 and Hive 2.3.7DAP-40895

Datameer X now supports Hadoop version 2.7.7 and Hive version 2.3.7.


7

Workbook: Add ability to 'update' instead of 'replace' for 'Cascade Permissions'DAP-40916

A user can now add permissions within 'Cascade Permissions' from a folder in the File Browser instead of fully replace the existing permissions.


8

Google BigQuery: Do not use temporary table when streaming from tablesDAP-40939

Imports from Google BigQuery now work without changes for tables, views and queries.


9

Plug-ins: JDBC/Snowflake - Unimplemented date/ time/ timestamp column splittingDAP-40952

The plug-in 'plugin-jdbc-v2' is now adapted to the column splitting needs.


10

Supported Hadoop Distributions: Add support for Cloudera CDP-DC/ Cloudera Runtime 7.1.2DAP-40998

Datameer X now supports Cloudera CDP-DC/ Cloudera Runtime version 7.1.2.


11

Supported Hadoop Distributions: Add support for Cloudera CDP-DC/ Cloudera Runtime 7.2.0.2DAP-40999

Datameer X now supports Cloudera CDP-DC/ Cloudera Runtime version 7.2.0.2.



Bug fixes

1

Workbook: SQL - 'EXCEPT' and 'INTERSECT' operator null value handlingDAP-40739

The Datameer functions 'EXCEPT' and 'INTERSECT' work properly again. The null value handling returns an empty result as expected.

(tick) Bug fixed.

2

Workbook: Validate for sheet error when changing column type which is used for an encoding sheetDAP-40892

The sheet error is being registered and a proper error message is shown when changing the column type which is used for an encoding sheet.

(tick) Bug fixed.

3Connection: ZooKeeper - Datameer does not close ZooKeeper connections (HBase)DAP-40970

The HBase plug-in was fixed so Datameer now closes ZooKeeper connections to HBase again when the connection is saved.

(tick) Bug fixed.

4

Connection: MariaDB dialect is missing export transformation 'SqlDateTransformation'DAP-40989

The needed export transformation is now added. MariaDB is changed to apply the export transformation 'SqLDateTransformation'.

(tick) Bug fixed.

5

Plug-ins: JDBC 2/ Snowflake - Integer column splitting skips table rows for importDAP-40990

All splitted columns from type integer can be imported completely.

(tick) Bug fixed.

6

Workbook: Sequential triggering does not respect the execution orderDAP-41000

Sequential triggering executes the artifacts based on the execution order.

(tick) Bug fixed.



Section

Datameer X 11.0.1

New Features

1

Workbook: New call to insert usage of sheets into Workbook config and details pageDAP-40722

A new workbook query includes the usage of a sheet. The new call includes the information and displays it in the workbook configuration and workbook details page. Therefore a user knows if a sheet must be kept or not.



Improvements

1

Workbook: Show each column in error state separately/ more prominentDAP-40370

Columns in an error state are highlighted in an error color in the columns list inside the Workbook Sheet Inspector. Clicking the column in error state highlights the column in the sheet. Furthermore the column list height is resizable now.


2

Workbook: Improve filter error validation by showing why it does not validateDAP-40594

When entering an invalid advanced filter definition, the error message states the problem of the filter definition.


3

Workbook Health Check: Provide the possibility to hide the loading screen and cancel the checkDAP-40607

You can now hide the Workbook Health Check loading screen and continue working while the check is executed in the background. Furthermore you can cancel the check completely.


4

Security: SAML authentication - role mapping proceedingDAP-40742

In the 'Admin' tab in the SAML authentication you can now assign a role and/ or a group to a user directly, e.g. 'User1: ANALYST:sales'. Note that the group must exist when assigning the group to the role.


5

Logging: New log file to store Datameer PID (Process ID)DAP-40761

A new log file named 'start-pid.log' is introduced to log the Datameer start time and the corresponding process ID. Whenever Datameer starts, a new line is being added to the log file. Find the log file under '/logs/dir'.



6

Workbook: Add join sheet creation validation queryDAP-40784

If an invalid join definition is trying to be created, a join validation provides a hint to the user why the join can not be executed.


7

Workbook: No consumer indication in the workbook configuration dialog if sheet is configured to be synced via Open Document FormatDAP-40818

The workbook configuration dialog now shows an indication that also synced sheets have a dependency consumed by. A workbook detail page contains truncated entries and a tooltip if the tables are longer than the rendered column width.


8

Security: Make a SAML plug-in aware of a load balancer that offloads SSLDAP-40868

The load balancer offloads SSL to the Datameer SAML plug-in. A new field is provided that enables - in addition to the existing Jetty-based SSL termination - the load balancer-based SSL offloading.


9

Add Support for Amazon EMR 5.30.1DAP-40875

Datameer now supports Amazon EMR 5.30.1


10

Security: SAML - Add option to read roles only from repositoryDAP-40886

User roles can now be read from the repository. All SAML assertion properties (username, email, groups and roles) are configurable as well.


Bug fixes

1

Export: Can't export string values into MariaDBDAP-38661

New Hadoop configuration parameters, that may be set in the 'Admin' tab → 'Cluster Configuration' section or during the export job configuration, override the dialect's default varchar field length. The export to MariaDB is now possible.

(tick) Bug fixed.

2

Import: Google BigQuery - Table list is empty after selecting a datasetDAP-40542

The table list appears not empty any more in the import job configuration.

(tick) Bug fixed.

3

Datameer Distribution: 'Snappy-java-*.jar' is missed in installation packages for Amazon EMR 6.0.0DAP-40721

'Snappy-java-1.1.4.jar' is now bundled with the distribution.

(tick) Bug fixed.

4

Housekeeping: Set flag to DATA_DELETION_TURNED_OFF instead of DELETION_FAILED when Housekeeping data deletion is disabledDAP-40744

The status column in 'filesystem_artifact_to_delete table can now have the value '2=DATA_DELETION_DISABLED'. That means that artifacts are scheduled for deletion but the distributed storage artifact deletion has been disabled via the 'housekeeping.data.deletion.disabled=true' flag.

(tick) Bug fixed.

5

Workbook: Columns.unequalCount error in join sheetDAP-40753

An error is shown when submitting a join definition with duplicate keys also stating how to recover. Find more information here.

(tick) Bug fixed.

6

Workbook: IllegalArgumentException for join operation with multiple conditionsDAP-40754

Join operations with multiple conditions work again.

(tick) Bug fixed.

7Connections: Databricks import connector throws an 'org.apache.spark.sql.AnalysisException'DAP-40764

The exception is fixed and the import works as expected.

(tick) Bug fixed.

8

Datameer distribution: Show port after starting a Datameer distribution with 'bin/conductor.sh startDAP-40765

The port is now displayed after starting Datameer.

(tick) Bug fixed.

9

Admin: Non-admin user can change folder ownership and select other users with the same roleDAP-40767

If a user has no capability 'Change Folder Owner' but the capability 'User can modify every file and folder' he is not able to change the owner of the folder.

(tick) Bug fixed.

10

Import: Columns of UUID type are being dropped from a PostgreSQL importDAP-40794

The UUID type is now supported for the import and therefore the import job is executed successfully.

(tick) Bug fixed.

11

Workbook: Can't duplicate or copy a workbook if the sheet execution order does not match the view orderDAP-40805

Duplicating or copying a workbook works as expected and Datameer follows the sheet logical order regardless of their current sequential position.

(tick) Bug fixed.

12

Database drivers: The MariaDB database driver has a wrong name in the JDBC connection string templateDAP-40855

The MariaDB database JDBC connection string now has the correct string template and you can establish a connection as expected.

(tick) Bug fixed.

13

Authentication: The default LDAP user filter does not filter out objects with username or email attribute set as an empty stringDAP-40884

User objects with a username or email attribute set to an empty string are now filtered out and do not break the LDAP cache creation.

(tick) Bug fixed.

14

Datameer distribution: Amazon EMR should use a conform Hive versionDAP-40890

A Datameer distribution with Amazon EMR now uses Hive 2.3.2.

(tick) Bug fixed.

15

Datameer distribution: Amazon EMR versions should use patched Hive/ Hadoop jars if available

DAP-40891

Datameer uses patched Hive and Hadoop jars for compatibly reasons.

(tick) Bug fixed.