Page Comparison

Info

title	INFO

Within these release notes Datameer X provides information and resolutions on improvements, bug fixes, and visual changes for 10.1.x maintenance release versions.

Visit Datameer's Help Center for additional information or contact your customer service representative.

Table of Contents

...

Datameer X 10.1.16

Improvements

...

1

...

Info

title	INFO

Within these release notes Datameer X provides information and resolutions on improvements, bug fixes, and visual changes for 10.1.x maintenance release versions.

Visit Datameer's Help Center for additional information or contact your customer service representative.

Table of Contents

Section

Datameer X 10.1.23

Bug Fixes

1	Authentication: SAML authentication - Invalid User Role Exception	DAP-42318
	An active Datameer X user can always log in regardless of the permissions assigned to the role.	Bug fixed.

2	Import: HBase Plug-in - Connection/ Import Job failure	DAP-42315
	After improving the HBase plugin, the connection as well as Import Jobs are working again.	Bug fixed.

3	External Systems: Incorrect mapping for BIG_DECIMAL columns	DAP-42309
	Datameer X BIG_DECIMAL columns are not supported for Hive table mapping when a workbook sheet is synced via the External System feature to Hive. For that, the Hive table must not contain BIG_DECIMAL columns.	Bug fixed.

Section

Datameer X 10.1.22

Bug Fixes

1	Workbook: Variables are highlighted red in the Formula Editor	DAP-42290
	Variables are not highlighted in red in the Formula Builder unless there is a syntax error.	Bug fixed.

2	Security: Update event logging plug-in regarding log4shell/ log4j and retire RTA plug-in	DAP-42307
	The event logging plug-in has been fixed regarding the log4shell functionality. Furthermore the RTA plug-in has been retired.	Bug fixed.

Section

Datameer X 10.1.21

Improvements

1	Supported Hadoop Distributions: Add Support for Cloudera Private Cloud/ Cloudera Runtime 7.1.7	DAP-42296
	Datameer X now supports Cloudera CDP/ Cloudera Runtime 7.1.7.

Bug Fixes

1	Security: Log4j v2 security breach (CVE-2021-44228)	DAP-42300
	Datameer X now has added {nolookups} tag to log output patterns of log4j v2.	Bug fixed.

2	Export: Cluster execution of PowerBI export jobs fail	DAP-42281
	A PowerBI export now succeeds while executed as job executed on the cluster.	Bug fixed.

3	Export: Datameer database export jobs randomly fail because of HDFS permissions issue	DAP-42275
	Datameer export jobs now work reliable again in secured cluster environments.	Bug fixed.

4	Execution Engine: Job that processes a broken GZIP file never completes	DAP-42269
	Datameer X now handles corrupted GZIP compressed files gracefully and terminates after importing all readable records.	Bug fixed.

Section

Datameer X 10.1.20

Improvements

1	Security: Implement Proxy for Azure Blob Storage (wasb/ wasbs)	DAP-42265
	You can now implement properties for routing wasb/ wasbs traffic from the Datameer Application server and Data Nodes via a proxy server when connecting to the Azure Blob Storage.

2	Supported Hadoop Distributions: Add support for Amazon EMR v6.4.0	DAP-42268
	Datameer X now supports Amazon EMR version 6.4.0.

Bug Fixes

1	Datameer on Amazon EMR: Job logs are accumulated in the local temp folder before pushed to Amazon S3	DAP-42271
	The customer switched to Hadoop’s S3A protocol for the EMR grid mode & metadata reads/ writes, log streaming etc. The DatameerS3 protocol is still in use for imports/ exports due to backward compatibility and all various DatameerS3 & Amazon S3 connection settings (region, endpoint, IAM with assumed role mode, …).	Bug fixed.

Section

Datameer X 10.1.19

Bug Fixes

1	Security: 'SSLHandshakeException' - Protocol is not appropriate	DAP-42262
	Creating a SSL-secured IMAP connection can now be established after updating the libraries.	Bug fixed.

Section

Datameer X 10.1.18

Bug Fixes

1	Execution Engine: Remove Spark execution engine from the Job Graph	DAP-42202
	Spark has been removed from the list of available execution frameworks from the Job Details page's Job Graph section. The Job Graph now shows only the available execution frameworks job durations.	Bug fixed.

Section

Datameer X 10.1.17

New Features

1	Admin: Job Scheduler - Allow to bypass the Job Scheduler queue	DAP-41875
	If needed, you can now execute priorized import and export jobs by bypassing the Job Scheduler's queue. For that, according role capabilities are needed as well as cluster resources.

Improvements

1	REST API: Provide REST API commands to pause and resume the Job Scheduler and auto compaction	DAP-36390
	You can pause and resume both the Job Scheduler and auto compaction via REST API commands.

Bug Fixes

1	Import: Google BigQuery - Datameer X can not convert Google BigQuery DATE to Datameer X DATE format	DAP-42220
	When importing data from Google BigQuery to Datameer X, the correct pattern is used to convert from Google BigQuery DATE format to Datameer X DATE format.	Bug fixed.

2	Workbook: Display the autocomplete list for the current sheet and column name	DAP-42180
	When using a formula that contains the current sheet name and the column name, the autocomplete list is now displayed.	Bug fixed.

3	Connection: Twitter REST API - Can't get token, improve error message	DAP-42108
	If a user tries to configure a Twitter Rest API and pressing the 'Authorize Datameer to retrieve data' button results in an error, now the actual error message is shown.	Bug fixed.

4	Workbook: 'Close Workbook' dialog appears every second time when closing an existing Workbook with no changes	DAP-42044
	When closing an open Workbook, in which no changes were applied, the 'Close Workbook' dialog is not displayed at no time.	Bug fixed.

5	Import/ Export: PowerBI - Creating a connection and export job fails	DAP-41634
	A user can now create a connection to PowerBI again. Configuring and running export jobs is possible again as well.	Bug fixed.

6	REST API: Curl command for uploading a license should not create a user session	DAP-41581
	Using the curl command for uploading a license does not increase the session count for user sessions any more.	Bug fixed.

7	REST API: Curl command for uploading a license is not working	DAP-41356
	The curl command for uploading a license is working as expected again.	Bug fixed.

Section

Datameer X 10.1.16

Improvements

1	Supported Hadoop Distributions: Add support for Amazon EMR v6.3.0	DAP-42063
	Datameer X now supports Amazon EMR version 6.3.0.

2	Supported Hadoop Distributions: Add support for Amazon EMR v5.33.0	DAP-42064
	Datameer X now supports Amazon EMR version 5.33.0.

3	Import: Oracle Eloqua - Importing Data into Datameer X	DAP-42077
	You can now ingest import data from Oracle Eloqua into Datameer X to receive campaign responses. For that, a JDBC driver is needed.

Bug Fixes

1	Workbook: Repoiting - 'NullPointerException' if the column to be repointed is blank	DAP-42141
	When repointing columns from a source sheet to a duplicated sheet, there is no 'NullPointerException' error anymore.	Bug fixed.

2	Upgrading Datameer X: Upgrade to Datameer X version 10.1.9 is blocked	DAP-42131
	After fixing a swallowed SQL sheet and fixing some minor dependencies, the upgrade could be executed successfully.	Bug fixed.

3	Workbook: Autocomplete in the 'Formula Builder' offers columns from all sheets	DAP-42105
	When inserting a formula in the Workbook's 'Formula Builder' only the sheets and columns for the first reference and only the columns from the current sheet are offered for autocompletion. Furthermore, autocomplete filters out dummy columns, e.g. insert a formula in the second column, apply, and then go to the first column.	Bug fixed.

4	Import/ Export: Databricks connection has the Export option under the Connection usage while it is not supported	DAP-42073
	The Connection usage list for a Databricks connection now contains only the connection type that is supported by the chosen connection type: Import.	Bug fixed.

5	Workbook: Renaming a column with a long name leads to a shift between the column header and the column	DAP-41921
	After renaming a column name, no shift is visible between the column header and the column.	Bug fixed.

Section

Datameer X 10.1.15

Improvements

1	Workbook: Provide an option to cancel the 'Repoint Sheet' dialog.	DAP-42003
	You can now cancel the 'Repoint Sheet' dialog after first applying a formula and then saving the sheet name. The applied formula is also being reset after cancelling the dialog.

Bug Fixes

1	Workbook: Workbook still appears in the tab bar after closing the Workbook	DAP-42027
	After closing a Workbook, the associated Workbook tab is being closed in the tab bar as well.	Bug fixed.

2	Import: JSON import - Datameer ignores the first array object	DAP-41996
	All arrays of the JSON file are being imported. With the 'include root array' property, Datameer parses the whole array including the first object also. In general when importing JSON: If a JSON file contains a 'root array', means the first level if the data structure is a JSON array (file starts with [...]), the 'include array' checkbox has to be checked during the import wizard. If a JSON file contains an object in the first level and a nested data structure contains an array for the data ingest (file starts with {...[...]...}), the 'include root array' must not be checked. Then the JSON path to the 'target array' for the data ingest must be entered into the textbook during the import wizard.	Bug fixed.

3	Datameer Distributions: Azure Blob Storage Connector - Packaging issues	DAP-41979
	After updating the core libraries, the Azure Blob Storage connector is working as expected again.	Bug fixed.

4	In-product documentation: Wrong documentation link for 'New Connection'	DAP-41944
	A user who wants to find out about more information about the 'Data Types' during the 'New Connection' workflow, is now directed to the correct public documentation page.	Bug fixed.

...

Section

Datameer X 10.1.14

New Features

1	Supported Hadoop Distributions: Add support for Google Dataproc version 1.5	DAP-40353
	Datameer X now supports Google Dataproc version 1.5.

Improvements

1	Supported Hadoop Distributions: Add support for Cloudera Private Cloud/ Cloudera Runtime 7.1.6	DAP-41823
	Datameer X now supports Cloudera Private Cloud/ Cloudera Runtime 7.1.6.

2	Admin Plug-ins: ‘plugin-tez’ - Upgrade to TEZ 0.10	DAP-41279
	After a plug-in patch, the 'plugin-tez' plug-in is now available for Datameer X distributions again.

3	Properties: AWS S3 http client max pool size should be configurable	DAP-41912
	The default maximum connection execution amount is now set to 100. With custom properties, you can change this amount to a higher or lower number. The custom count can also be set via Hadoop property in the cluster settings.

4	Workbook: Create Toolbar button to auto-expand all column width to make all column names visible	DAP-41603
	Datameer has now a new toolbar button to expand all column width to make all column names fully visible.

5	Workbook: Formula sheet repointing	DAP-41549
	When changing a formula sheet that points to another sheet, Spectrum now detects this changes and displays a dialog: The user can apply the changes to only the current column or to all columns. This dialog is only displayed, when when more than one column is affected.

Bug Fixes

1	Workbook: Parquet format - Ingesting creates duplicate 'das' metadata columns	DAP-41874
	When ingesting a Parquet file into Datameer by a File Upload, Import Job or Datalink, there will be no more duplicated columns displayed in the tab "Define Fields".	Bug fixed.

2	Snowflake Export: Tables are not updating when selecting another schema	DAP-41721
	When configuring a Snowflake export or import job, the tables are now updated when changing the schema from the drop-down.	Bug fixed.

3	Workbook synchronisation: Sheet synchronization - Validation error is still showing even after update	DAP-41476
	After a formerly incorrect formula has been changed into a valid formula, the validation error is no longer displayed.	Bug fixed.

4	FileBrowser: Remove the 'Import Job' tab from the tab bar after the job has been deleted	DAP-41635
	The 'Import Job' tab is now deleted from the tab bar and is no longer displayed, after the job has been deleted.	Bug fixed.

...

Section

Datameer X 10.1.13

New Features

1	Export: Amazon Redshift incremental export	DAP-35849
	Incremental export to Amazon Redshift is now available when configuring an export. For that, select "Incremental" as the Record Replacement Policy and select the required column from the dataset.

Improvements

1	Supported Hadoop Distributions: Add support for Cloudera Private Cloud/ Cloudera Runtime 7.1.5.14	DAP-41578
	Datameer X now supports Cloudera Private Cloud/ Cloudera Runtime 7.1.5.14.

Bug Fixes

1	Workbook: Workbook toolbar buttons are not working when using them in the Internet Explorer	DAP-41864
	The Datameer X Workbook toolbar now works with Internet Explorer.	Bug fixed.

2	Export: Amazon Redshift (Fast Load) export is dropping records	DAP-41838
	The data export into Amazon Redshift (Fast Load) is working correctly again.	Bug fixed.

3	Workbook: Workbook toolbar buttons are not working when using them in the latest Chrome version	DAP-41836
	The Datameer X Workbook toolbar now works with Chrome version 89.0.4389.90 (Official Build) and newer.	Bug fixed.

4	Execution Engine: Datameer X on Amazon EMR/ Google Cloud Platform - Failed to activate the plug-in 'plugin-dwh'	DAP-41816
	Since 'plugin-doh' is a conductor-only plug-in, it is not being copied to the cluster anymore and does not impact any job execution.	Bug fixed.

5	Import/ Export: Snowflake - Tables are not updating when selecting another schema	DAP-41721
	When configuring a Snowflake export or import job, the tables are now updated when changing the schema from the drop-down.	Bug fixed.

6	User Interface Login: Intermittently login page is not refreshing after a successful login	DAP-41633
	The correct login is working in all browsers for SAML, internal authentication as well as for external authentication (LDAP).	Bug fixed.

...

Section

Datameer X 10.1.12.2

Bug Fixes

1	Workbook: Workbook toolbar buttons are not working when using them in the Internet Explorer	DAP-41864
	The Datameer X Workbook toolbar now works with Internet Explorer.	Bug fixed.

2	Workbook: Workbook toolbar buttons are not working when using them in the latest Chrome version	DAP-41836
	The Datameer X Workbook toolbar now works with Chrome version 89.0.4389.90 (Official Build) and newer.	Bug fixed.

...

Section

Datameer X 10.1.12

New Features

1	Supported Hadoop Distributions: Add support for Amazon EMR 6.2.0	DAP-41575
	Datameer X now supports Amazon EMR 6.2.0.

2	Supported Hadoop Distributions: Add support for Amazon EMR 5.32.0	DAP-41574
	Datameer X now supports Amazon EMR 5.32.0.

Improvements

1	Workbook: Auto-expand column header during "column edit" if the column name is partly invisible	DAP-41677
	When renaming a column, the column width is now expanded fully visible without any truncation.

2	Supported Hadoop Distributions: Enable Apache Hadoop 2.8.5 support	DAP-41451
	Datameer X now supports Apache Hadoop 2.8.5, also with Hive 2.3.8 as well as HBase 1.4.13.

Bug Fixes

1	Export: Snowflake - Export jobs failing with exception	DAP-41557
	Snowflake export jobs run successfully again after fixing the plug-in.	Bug fixed.

2	File Browser: Dependencies View artifact loses its color after selecting/ deselecting	DAP-41322
	In the Dependencies View a selected and then deselected artifact does not change the color to grey anymore.	Bug fixed.

...

Section

Datameer X 10.1.11

New Features

1	Datameer Service: New Event Logging Plug-in	DAP-41323
	The new Event Logging plug-in allows to log an artifact creation or deletion event. Certain types can be enabled and disabled for logging them. If an event type is not checked, no entry will be logged if the action happens in Datameer X. The log file is then downloadable with the System and Usage Report.

Improvements

1	Workbook: Auto-expand column header during "Column edit" if the column name is partly invisible	DAP-41677
	When renaming a column, the column width is now expanded fully visible without any truncation.

2	Supported Hadoop Distributions: Add Support for Cloudera CDP-DC/ Cloudera Runtime 7.1.4.1	DAP-41067
	Datameer X now supports Cloudera CDP-DC/ Cloudera Runtime 7.1.4.1.

Bug fixes

1	Export: Snowflake - Invalid 'DATE' type export	DAP-41329
	The Datameer 'DATE' values are displayed properly as Snowflake's 'TIMESTAMP' values or other data-like data types when exporting.	Bug fixed.

2	Performance: JDBC-based Data Link very large sample jobs do not limit the records, read and run too long on large tables	DAP-41334
	Due to performance improvements, JDBC-based Data Link sample jobs finish faster and collect 5,000 sample records by default.	Bug fixed.

3	Service: Conductor.sh Check triggers tmp/ directory clean up	DAP-41336
	The tmp/ directory clean up is not triggered while Datameer X is running any more. The Conductor.sh Check command does not trigger the tmp/ directory clean up.	Bug fixed.

4	Workbook: Pivot Sheet throws and NullPointerException in case its source sheet has no records	DAP-41620
	When a source of a Pivot sheet has no records, an error is now displayed in the Pivot sheet that there is no data in the source sheet.	Bug fixed.

5	Connections: Amazon S3 - IAM import/ export failure	DAP-41667
	After replacing the jar files the import and export are working as expected again.	Bug fixed.

6	File Browser: Dependencies View artifact loses its color after selecting/ deselecting	DAP-41322
	In the Dependencies View a selected and then deselected artifact does not change the color to grey anymore.	Bug fixed.

7	Workbook: SQL sheets - TRIM function in WHERE clause	DAP-41380
	After applying the LIKE clause the TRIM function returns the required records accordingly.	Bug fixed.

...

Section

Datameer X 10.1.10

New Features

1	Import/Export: Google BigQuery - Add partition filtering functionality	DAP-41202
	It is now possible to import from a partitioned table.

2	Workbook: Custom functions for dictionary matching	DAP-41237
	Datameer X has now two new functions for dictionary matching: "MATCHING_DICTIONARY" and "MATCHING_DICTIONARY_FUZZY".

3	Import/ Export: S3 connection - Add the IAM role ARN option	DAP-41298
	There is a new authentication option for the S3 connection: You can now authenticate via the IAM role with an assumed role.

Improvements

1	Plug-Ins: plugin-hive - do not overwrite the fetch size and use the value defined in the jdbc connection string	DAP-41130
	The fetch size is new increased up to 5,000 to boost the jdbc-based import jobs.

2	Authentication: Terminate all user sessions on logout	DAP-41159
	Datameer X presents a new property 'devicesconductor.session.management.enabled'. If it is set to true, a second login with the same user will be prohibited. The property is disabled by default.

3	Admin UI: Database driver tab - wide grid or horizontal scrollbar	DAP-41197
	The UI has now more screen space for the grid, showing the installed JDBC drivers. Resizing the File Browser window, the width of the table is being adjusted.

4	Export: Hive - Improve the help text for the replacement strategy during the Hive export	DAP-41223
	The user now sees a more specific help text for the replacement strategy for a Hive export.

Bug fixes

1	File Browser: Disable the folder context menu 'Rename' in the Folder Tree	DAP-40713
	If no folder is selected within the Folder Tree the 'Rename' option from the context menu will be disabled.	Bug fixed.

2	File Browser: Wrong documentation link for the file search	DAP-40889
	When searching for files in the File Browser the help link now points to the correct corresponding documentation page.	Bug fixed.

3	Workbook: Wrong color highlighting of formula parameter in the Formula Editor	DAP-40933
	Formula highlighting works as expected again.	Bug fixed.

4	Workbook: Scheduling information is shown on a Workbook Details page even when the scheduling is disabled	DAP-41097
	The scheduling information from the Workbook Details page is now up-to-date after changing the scheduling manually.	Bug fixed.

5	Datameer Distribution: Jars missing in installation package for CDH-7.x.x	DAP-41180
	The installation package for Cloudera CDH-7.x.x distributions is now complete.	Bug fixed.

6	Connection: 'Save Connection' dialog - 'Close' is visible under the 'close' icon	DAP-41205
	The string 'close' within the 'Save Connection' dialog does not appear anymore.	Bug fixed.

7	Workbook: Post Upgrade Issue - NullPointerException in partition filter with $lastestpartition	DAP-41222
	Datameer X will not throw a NullPointerException anymore if a user tries to update the partition filter in a Workbook.	Bug fixed.

8	REST API: Concurrent session management blocks REST API to work	DAP-41242
	The concurrent session feature still works when enabled. The REST API works independently from enabling or disabling the concurrent session feature.	Bug fixed.

9	File Browser: Dependencies View only opens after the second attempt	DAP-41291
	Clicking the 'Dependencies View' icon now opens the Dependencies View at the first try.	Bug fixed.

10	Workbook: Attempt to duplicate the Workbook leads to broken references in the new artifact	DAP-41297
	Duplicating or copying an artifact does not lead to broken references in the new artifact anymore.	Bug fixed.

Section

Datameer X 10.1.9

Improvements

1	Workbook Health Check: Log status and progress into separate logs file 'logs/workbook_health.log'	DAP-41120
	The Workbook Health Check logs are now being written into the file 'logs/workbook_health_check.log'.

2	Supported Hadoop Distributions: Add support for Cloudera CDH 6.3.4	DAP-41138
	Datameer X now supports Cloudera CDH 6.3.4.

3	Plug-ins: 'Plugin-dwh' - Make Data Warehouse plug-in default for Datameer distribution	DAP-41150
	The plugin 'plugin-dwh' is now available as a default plugin and comes automatically with all Datameer X distributions.

Bug Fixes

1	File Browser: After canceling 'Create new folder', the File Browser does not update when selecting another folder	DAP-40673
	After canceling 'Create new folder', it is possible to change the folder in the Folder Tree panel.	Bug fixed.

2

Pivot Sheet: Creating a Pivot sheet with a broken Workbook source ends up in an error dialog

DAP-40729

Creating a Pivot sheet out of a broken Workbook is not possible since the error state of a broken Workbook is shown first before one can create a sheet out of a broken Workbook.

Bug fixed.

3	Workbook Details: The table for SheetDependent is missing/ not displayed	DAP-40821
	The table for 'SheetDependent' ("Consumed by x artifacts") is visible on the Workbook Details page.	Bug fixed.

4	File Browser: Prevent the option to open the 'Job run details' page if the necessary capability is missing	DAP-41129
	If a user does not have the capability “Common job details“ he will not enter the job-execution details page and an error message is shown instead.	Bug fixed.

5	Import: Clickhouse - Ingest of empty tables is not working	DAP-41140
	Ingesting empty tables now works properly when pointing to an empty table in a Clickhouse database.	Bug fixed.

Section

Datameer X 10.1.8

New Features

1	Import/ Export: Enable Import to ClickHouse via a JDBC connection	DAP-40951
	Importing data via a JDBC connection from a ClickHouse database is now possible. You need to have the ClickHouse database driver installed and then configure the JDBC connection to execute imports and exports.

2	Supported Hadoop Distributions: Add support for Amazon EMR 5.31.0	DAP-41025
	Datameer now supports Amazon EMR version 5.31.0.

3	File Browser: Add new root folder in the Folder Tree panel	DAP-41033
	A user can now add a new top-level folder/ root folder in the Folder Tree panel in the File Browser. The newly created folder is named 'Untitled Folder' and can be renamed immediately after creating it. The capability 'CREATE_FOLDER' is needed.

Improvements

1	Workbook Health Check: Enable the download of the the report and print the configuration IDs of all broken Workbooks	DAP-40628
	You can now download the Workbook Health Check as a JSON from the Workbook Health Check page. Furthermore the configuration IDs of all broken Workbooks are displayed as a comma-separated list unter the Workbook Health section.

2	Rebranding: Datameer Classic is now DatameerX	DAP-40913
	In course of the rebranding, The former 'Datameer Classic' is now called 'DatameerX'. The new Datameer products are called 'Spectrum' and 'Spotlight'. Find more information about all of our products on our website.

3	SFTP/ SCP import: Reduce the amount of the opened connections while multiple small file import	DAP-41011
	The new configuration property 'das.cluster.hadoop.fs.cache.scheme.whitelist' which is configurable globally and on job level, reduces the amount of SFTP/ SCP sessions. With the FileServer cache enabled for scp/sftp scheme, there will be a single SSH connection for each cluster task launched, limiting the "connection refused" occurances.

4	Authentication: SAML authentication - Disable the local account management	DAP-41058
	The mixed-mode parameter that defines whether or not local authentication is available while using SAML (or LDAP), can now be configured within the SAML plug-in settings. Having it set to false, only the external authentication mode is available. Having it set to true, both external and local authentication mode are available.

5	Connection: Amazon S3 Native connector should validate the region and bucket combination properly	DAP-41074
	When setting up an Amazon S3 Native connection the region is now validated against the bucket properly.

6	Housekeeping Service: Housekeeping Service should attempt to delete HDFS data artifacts for non-existent users	DAP-41090
	Deleting artifacts that are owned by a removed Datameer user is now possible. Deleting can be executed a Datameer Service user that has direct access to the HDFS cluster.

Bug Fixes

1	User Interface: Scrolling problems when the browser is vertically maximized	DAP-40723
	When the browser window is vertically maximized, the scroll bar works as expected and you can scroll up and down within a Workbook.	Bug fixed.

2	Sequential triggering: Do not display a complete list of all variables on the import/ export job run details page	DAP-41007
	The list 'Variables' from the import/ export job run details page contains only the used variables.	Bug fixed.

3	File Browser: Unable to 'Duplicate Workbook', the dialog remembers the previous filename value	DAP-41020
	Duplicating a Workbook works as expected. Executing the duplication creates a copy of the file and can be renamed as required. The default name of a duplicated Workbook is 'Copy_of_<initial Workbook name>'.	Bug fixed.

4	Connection: Google BigQuery export fails with staging bucket name that contains underscores	DAP-41045
	It is now possible to have the Google Cloud Storage bucket containing underscores as the Google BigQuery connector staging bucket/ directory.	Bug fixed.

5	Login: After session timeout - error when first re-login attempt	DAP-41054
	A user is able to log in to the Datameer User Interface after a session timeout from the first attempt on.	Bug fixed.

6	Security: Datameer Support Engineer Report extension allows reading random files from the Datameer application server OS	DAP-41059
	The Datameer extension Support Engineer Report reads files from the Datameer installation folder only (configuration as well as log files).	Bug fixed.

7	REST API: Path information disclosure when calling for a non-existent file	DAP-41060
	When calling for a non-existent file a '404' error is shown containing the requested file name.	Bug fixed.

8	Security: 'productDASData' text/ javascript block is available at the page's source even when the 'About' menu is disabled	DAP-41061
	When the 'About Datameer' menu is disabled, the 'productDASData' text/ javascript block is deactivated as well.	Bug fixed.

9	Plug-ins: plugin-s3 (S3 Native connector) on EMR - Export to a bucket in different S3 region fails	DAP-41073
	The plug-in is fixed. An export via the S3 Native connection works regardless of a bucket region.	Bug fixed.

10	Export: An export job should not inject 'DataStoreModel#setupConf' into the global 'JobConf'	DAP-41088
	An export job creates a dedicated 'JobConf' for export-only purposes.	Bug fixed.

11	Authentication: EMR- S3 connection uses IAM mode always when the cluster mode S3 is set to IAM	DAP-41100
	The Amazon S3 connection uses the configured authentication mode from the connection configuration: Access Secret/ Key.	Bug fixed.

Section

Datameer X 10.1.7

New Features

1	Supported Hadoop Distributions: Add support for Amazon EMR 6.1.0	DAP-40876
	Datameer X now supports Amazon EMR version 6.1.0.

Improvements

1	Import/ Export: Support for Azure Synapse (formerly Microsoft SQL Data Warehouse)	DAP-27739
	You can now import from or export to an Azure Synapse Database. The according dialect is being supported as well.

2	Supported Hadoop Distributions: Add support for Amazon EMR 5.30.1	DAP-40875
	Datameer X now supports Amazon EMR version 5.30.1.

3	Supported Hadoop Distributions: Add support for Cloudera CDP-DC/ Cloudera Runtime 7.1.3	DAP-40877
	Datameer X now supports Cloudera CDP-DC/ Cloudera Runtime version 7.1.3.

4	Connection: Rewrite Google BigQuery import connector to use Google BigQuery storage API	DAP-40883
	When importing from or exporting to Google BigQuery, the new Google BigQuery storage API is used for more efficient imports and exports..

5	Authentication: Remove 'gssapi-with-mic' from Datameer's 'default.properties'	DAP-40799
	The value 'gssapi-with-mic' is removed from the 'default.properties' key 'system.property.jsch.preferred.authentication'. The new added value is 'password,keyboard-interactive,publickey'.

6	Authentication: SAML - Add option to read roles only from repository	DAP-40886
	You can now configure the SAML authentication to read all roles from the repository. Furthermore all four assertion properties are configurable (username, email, groups, roles).

7	Supported Hadoop Distributions: Add support for Apache Hadoop 2.7.7 and Hive 2.3.7	DAP-40895
	Datameer X now supports Hadoop version 2.7.7 and Hive version 2.3.7.

8	Workbook: Add ability to 'update' instead of 'replace' for 'Cascade Permissions'	DAP-40916
	A user can now add permissions within 'Cascade Permissions' from a folder in the File Browser instead of fully replace the existing permissions.

9	Google BigQuery: Do not use temporary table when streaming from tables	DAP-40939
	Imports from Google BigQuery now work without changes for tables, views and queries.

10	Plug-ins: JDBC/Snowflake - Unimplemented date/ time/ timestamp column splitting	DAP-40952
	The plug-in 'plugin-jdbc-v2' is now adapted to the column splitting needs.

11	Supported Hadoop Distributions: Add support for Cloudera CDP-DC/ Cloudera Runtime 7.1.2	DAP-40998
	Datameer X now supports Cloudera CDP-DC/ Cloudera Runtime version 7.1.2.

12	Supported Hadoop Distributions: Add support for Cloudera CDP-DC/ Cloudera Runtime 7.2.0.2	DAP-40999
	Datameer X now supports Cloudera CDP-DC/ Cloudera Runtime version 7.2.0.2.

Bug Fixes

1	Workbook: SQL - 'EXCEPT' and 'INTERSECT' operator null value handling	DAP-40739
	The Datameer X functions 'EXCEPT' and 'INTERSECT' work properly again. The null value handling returns an empty result as expected.	Bug fixed.

2	Authentication: The default LDAP user filter does not filter out objects with username or email attribute set as an empty string	DAP-40884
	The corresponding plugin is patched. User objects with and username or email attribute set to an empty string are now filtered out and do not break the LDAP cache creation.	Bug fixed.

3	Workbook: Validate for sheet error when changing column type which is used for an encoding sheet	DAP-40892
	The sheet error is being registered and a proper error message is shown when changing the column type which is used for an encoding sheet.	Bug fixed.

4	Workbook: Inaccurate status and missing dialog if triggering the Workbook from configuration page	DAP-40915
	If triggering the Workbook from the Workbook Configuration page by the 'Starting calculation process immediately after save' option the process dialog is visible again and the status bar shows the accurate status.	Bug fixed.

5	Workbook: The key shortcut 'ALT + O' does not work	DAP-40923
	Executing the key shortcut 'ALT + O' opens the Workbook from the File Browser as expected.	Bug fixed.

6	Connection: ZooKeeper - Datameer X does not close ZooKeeper connections (HBase)	DAP-40970
	The HBase plug-in was fixed so Datameer X now closes ZooKeeper connections to HBase again when the connection is saved.	Bug fixed.

7	Connection: MariaDB dialect is missing export transformation 'SqlDateTransformation'	DAP-40989
	The needed export transformation is now added. MariaDB is changed to apply the export transformation 'SqLDateTransformation'.	Bug fixed.

8	Plug-ins: JDBC 2/ Snowflake - Integer column splitting skips table rows for import	DAP-40990
	All splitted columns from type integer can be imported completely.	Bug fixed.

9	Workbook: Sequential triggering does not respect the execution order	DAP-41000
	Sequential triggering executes the artifacts based on the execution order.	Bug fixed.

Section

Datameer X 10.1.6

New Features

1	Admin: Support for token authentication within the Tableau plug-in	DAP-40342
	You can now select the authentication mode 'SSO Token' when creating a connection to a Tableau Server.

2	Workbook: New call to insert usage of sheets into Workbook config and details page	DAP-40722
	A new workbook query includes the usage of a sheet. The new call includes the information and displays it in the workbook configuration and workbook details page. Therefore a user knows if a sheet must be kept or not.

Improvements

1	Workbook Health Check: Provide the possibility to hide the loading screen and cancel the check	DAP-40607
	You can now hide the Workbook Health Check loading screen and continue working while the check is executed in the background. Furthermore you can cancel the check completely.

2	Security: SAML authentication - role mapping proceeding	DAP-40742
	In the 'Admin' tab in the SAML authentication you can now assign a role and/ or a group to a user directly, e.g. 'User1: ANALYST:sales'. Note that the group must exist when assigning the group to the role.

3	Logging: New log file to store Datameer X PID (Process ID)	DAP-40761
	A new log file named 'start-pid.log' is introduced to log the Datameer X start time and the corresponding process ID. Whenever Datameer X starts, a new line is being added to the log file. Find the log file under '/logs/dir'.

4	Workbook: Search field and 'go to column' feature in the Workbook Sheet Inspector	DAP-40338
	A user can now search for a column in the Workbook Sheet Inspector and also go directly to the column by clicking it.

5

Workbook: Show each column in error state separately/ more prominent

DAP-40370

Columns in an error state are highlighted in an error color in the columns list inside the Workbook Sheet Inspector. Clicking the column in error state highlights the column in the sheet. Furthermore the column list height is resizable now.

6	Workbook: No consumer indication in the workbook configuration dialog if a sheet is configured to be synced via Open Document Format	DAP-40818
	The workbook configuration dialog now shows an indication that also synced sheets have a dependency consumed by. A workbook detail page contains truncated entries and a tooltip if the tables are longer than the rendered column width.

7	Security: Make SAML plug-in aware of a load balancer that offloads SSL	DAP-40868
	The load balancer offloads SSL to the Datameer X SAML plug-in. A new field is provided that enables - in addition to the existing Jetty-based SSL termination - the load balancer-based SSL offloading.

Bug Fixes

1	Export: Can't export strings values into MariaDB	DAP-38661
	New Hadoop configuration parameters, that may be set in the 'Admin' tab → 'Cluster Configuration' section or during the export job configuration, override the dialect's default varchar field length. The export to MariaDB is now possible.	Bug fixed.

2	Workbook: Error message isn't visible in the File Browser if trying to run an export-only workbook/ sheet without data	DAP-40440
	The error message that an export job of a workbook or sheet without data is failing, is now displayed again.	Bug fixed.

3	File Browser: 'Dependencies graph' performance	DAP-40727
	The performance for using the 'All Dependencies' and 'Depends On' function in the File Browser is now improved.	Bug fixed.

4	Workbook: Columns.unequalCount error in join sheet	DAP-40753
	An error is shown when submitting a join definition with duplicate keys also stating how to recover. Find more information here.	Bug fixed.

5	Workbook: IllegalArgumentException for join operation with multiple conditions	DAP-40754
	Join operations with multiple conditions work again.	Bug fixed.

6	Connections: Databricks import connector throws an 'org.apache.spark.sql.AnalysisException'	DAP-40764
	The exception is fixed and the import works as expected.	Bug fixed.

7	Admin: Non-admin user can change folder ownership and select other users with the same role	DAP-40767
	If a user has no capability 'Change Folder Owner' but the capability 'User can modify every file and folder' he is not able to change the owner of the folder.	Bug fixed.

8	Import: Columns of UUID type are being dropped from a PostgreSQL import	DAP-40794
	The UUID type is now supported for the import and therefore the import job is executed successfully.	Bug fixed.

9	Workbook: Can't duplicate or copy a workbook if the sheet execution order does not match the view order	DAP-40805
	Duplicating or copying a workbook works as expected and Datameer X follows the sheet logical order regardless of their current sequential position.	Bug fixed.

10	Database drivers: The MariaDB database driver has a wrong name in the JDBC connection string template	DAP-40855
	The MariaDB database JDBC connection string now has the correct string template and you can establish a connection as expected.	Bug fixed.

11	User Interface: Workbook icons are disabled for the own workbook	DAP-40871
	After the code was patched, the icons for a workbook are available and not greyed out anymore in the user interface.	Bug fixed.

Section

Datameer X 10.1.5

New Features

1	Database Support: Create Azure Cosmos database dialect	DAP-40590
	You can now connect Datameer X against Azure Cosmos. Therefore you need to configure Azure Cosmos DB as a connector and install the corresponding JDBC driver.

2	Supported Hadoop Distributions: Add support for MapR Extension Package 6.3 with MapR 6.1	DAP-40652
	Datameer X now supports MapR Extension Package 6.3 with MapR 6.1.

Improvements

1	Documentation: Help links shouldn't be shown in the UI when the property pointing to the documentation is invalid or empty	DAP-40386
	No links that are pointing to the Datameer X documentation are shown in the UI when the 'doc.url' property is not set. The 'Help' link in the context menu does not appear anymore.

2	Workbook: Add indicator in workbook configuration and workbook details page if a sheet is used as dependency	DAP-40444
	There are new indicators for sheets within a workbook whose sheets are either a source data or for an export. The indicator appears on both the workbook configuration page next to each sheet, as well as in the workbook's detail page.

3	Admin: Workbook Health Check improvements	DAP-40479
	You can now zoom in and out within the health check graph and navigate back to the original size. Furthermore you can view all broken workbooks in a expandable list within the workbook health report. The workbook IDs are clickable and point to the workbook node in the health graph. If a node is then selected it can be opened in the info panel.

4	Datameer X White Labeling: White labeling Datameer X for OEM	DAP-40550
	Several default properties were created to change the Datameer X UI and URL to have the Datameer X application white labeled. Furthermore the Datameer X distribution allows to change the small and big logo as well as favicons.

5	Performance: Optimize Browser Service WB#isSync calls for better performance	DAP-40565
	The workbook sync call is optimized to show the last sync and current sync status in the File Browser.

6

Workbook: Implement a workbook dependency cycle detector

DAP-40706

A new workbook dependency cycle detector to identify workbook dependency cycles is implemented on the '/dev' page. After all workbooks are checked, a HTML report is accessible. A workbook is checked for being part of a workbook cycle.

7	Performance: Change task order in Housekeeping Service	DAP-40711
	A new default property 'housekeeping.run.task-attempts-per-run' is set to a value of '50' per default to decrease the performance of the Housekeeping Service significantly when it is to low.

Bug Fixes

1	FileBrowser: Could not delete a file upload after saving a copy	DAP-39776
	Deleting a file upload after saving a copy now works again.	Bug fixed.

2	Workbook: If a user has both "view" and "edit" permissions they should be able to rename a folder	DAP-40544
	A user which has the new capability "Rename Folder" and edit permissions to the parent folder, he can rename a child folder within a parent's folder. Having only the "Rename Folder" permissions but not the permissions to edit the parent folder, the user can rename the folders that are located in the parent's folder. A user of a folder can always rename the folder's default settings.	Bug fixed.

3	ImportJob: Change links in 'Import Job Details' view to actual buttons	DAP-40382
	The button 'Browse Data' works as expected again.	Bug fixed.

4

Import/ Export: Can't connect to the SFTP server hidden behind a proxy

DAP-40401

Datameer X is now able to connect to a SFTP server that is hidden behind a proxy with the following properties:

No Format

#proxy settings to connect a SFTP server hidden by the proxy
das.network.proxy.sftp.type=[HTTP]
das.network.proxy.sftp.host=<host ip>
das.network.proxy.sftp.port=<port>
das.network.proxy.sftp.username=<username>
das.network.proxy.sftp.password=<password>

Bug fixed.

5	Workbook: Double-click to submit in credentials dialog can lead to an error	DAP-40487
	Adding a datasource to a session-based import job doesn't lead to an error any more.	Bug fixed.

6	Admin: Health Check report sometimes does not update correctly	DAP-40509
	When clicking the "Start" button from the Workbook Health Check the indicator now shows the correct status of the Workbook Health Check.	Bug fixed.

7	Datameer X Upgrade: When upgrading from Datameer X 6.4.13 to 7.5.10 several workbooks stuck at 'Registering operations for sheet' phase	DAP-40534
	When upgrading the workbooks don't stuck anymore because of patches in *.jar libraries, e.g. big workbooks with sophisticated logic.	Bug fixed.

8	Import Job: The partition dialog in 'Browse all Data' does not close when clicking "Select"	DAP-40536
	The dialog now closes again when clicking "Select".	Bug fixed.

9	FileBrowser: Browser list-files slow performance when a folder contains workbooks with data-driven scheduling	DAP-40544
	The backend query was optimized to minimize to response time of folders within the FileBrowser that contain artifacts with a long chain of upstream dependencies, e.g. workbooks that have 10 to 15 layers of parent artifacts.	Bug fixed.

10	Import Job: Fix intermittent failing test for email format validation	DAP-40551
	The test for the email validation now works as expected and detects not accepted email addresses, e.g. within an import job configuration.	Bug fixed.

11	Import: Azure Databricks import fails	DAP-40604
	An Azure Databricks import now works as expected.	Bug fixed.

12	Admin: Workbook Health Check is broken	DAP-40606
	The Workbook Health Check from the Admin tab now works properly again.	Bug fixed.

13	Export/ Hive: Data is being written as 'TEXT' format when duplicating an export job and setting a new table and data type	DAP-40655
	When duplicating a Hive export job and changing its settings, the changes are respected without any changes of data type, output format etc. in the new Hive table	Bug fixed.

14	Setup: Users without submitted license agreement can not operate Datameer	DAP-40681
	The license agreement flag 'force.license-agreement' is set to 'TRUE' and the internal user management is configured to allow newly created users to login and accept the terms of licence to operate Datameer.	Bug fixed.

15	Setup: C3p0 connection pool exhaustion	DAP-40687
	After disabling the statement caching no c3p0 warnings about pool problems appear anymore.	Bug fixed.

16	Performance: Housekeeping Service MySQL CPU drain	DAP-40604
	A new default property 'housekeeping.run.task-attempts-per-run' is set to a value of '50' per default to decrease the performance of the Housekeeping Service significantly when it is to low.	Bug fixed.

17	Workbook: Prevent Out of Memory errors when handling workbook circular dependencies	DAP-40694
	Out of Memory errors are now prevented by changing buggy code.	Bug fixed.

18	Workbook: Prevent Stack Overflow errors when handling workbook circular dependencies	DAP-40696
	Stack Overflow errors are now prevented by changing buggy code.	Bug fixed.

19	Performance: Dependencies graph performance is low	DAP-40727
	The performance for using the 'All Dependencies' and 'Depends On' function in the File Browser is now improved.	Bug fixed.

...

Section

Datameer X 10.1.4.1

Improvements

1	Datameer X White Labeling: White labeling Datameer X for OEM	DAP-40550
	Several default properties were created to change the Datameer X UI and URL to have the Datameer X application white labeled. Furthermore the Datameer X distribution allows to change the small and big logo as well as favicons.

2	Datameer X White Labeling: Help links shouldn't be shown if a property pointing to the public documentation is invalid or empty	DAP-40386
	Links to the Datameer X public documentation are not shown within the white labeled Datameer X UI by default. The links are only shown when the property 'doc url' is set. This is also valid for the 'Help' menu.

3	FileBrowser: Optimize BrowserService calls for better performance	DAP-40565
	The BrowserService is now optimized not to access "WB#getSheets" and "WB#isSync".

Bug Fixes

1	Upgrading Datameer X After an upgrade several workbooks stuck at the "Registering operations for sheet" phase.	DAP-40534
	Datameer X workbooks don't get stuck anymore.	Bug fixed.

2	Performance: Browser list-files show a slow performance when the folder contains workbooks with data-driven scheduling	DAP-40544
	Browser list-files work with regular performance when a folder contains workbooks with data-driven scheduling.	Bug fixed.

Section

Datameer X 10.1.4

Improvements

1	Supported Hadoop Distributions: Add Support for Cloudera CDP-DC/ Cloudera Runtime 7.1.1	DAP-40110
	Datameer X now supports Cloudera Runtime version 7.1.1.

2	Workbook: Change 'SHIFTTIMEZONE' function arguments' name	DAP-40199
	The description of the function arguments has been changed in the formula builder to prevent ambiguities. There is no behavior change of the functionality of the function itself.

3	Setup: Enable the 'DAS_USER' property in the 'etc/das-env.sh' file to reflect the assigned service account	DAP-40247
	Adjusting this parameter is critical in saving the Admin from themselves by preventing an accidental startup of Datameer X without first assuming the proper service account. The property 'DAS_USER' must be enabled in the 'etc/das-env.sh' file.

4	Workbook: Extend variable value length	DAP-40265
	Within a fresh Datameer X distribution, it is now possible to create variable values with more than 255 characters.

5	Performance: Validate Datameer X license status once a day only	DAP-40272
	A Datameer X license now gets checked with the first user authentication or scheduled job run after midnight (local time) and is valid 24 h afterwards.

6	Workbook: Validate cross joins for SQL sheets	DAP-40276
	Datameer X validates queries of SQL sheets and notifies an analyst if he accidentally created a cross join in a 'FROM' section. Furthermore Datameer X allows the explicit usage of a 'cross join' syntax. The validation can be enabled by setting the 'sql-cross-join.enabled property' in the 'default.properties' configuration file.

7	Setup: Document MariaDB as a supported database dialect for Datameer's metadata database	DAP-40293
	MariaDB is now listed as a valid setting for the 'system.property.db.mode'.

8	Supported Hadoop Distributions: Add Support for Amazon EMR v5.30.0	DAP-40355
	Datameer X now supports Amazon EMR v5.30.0.

9	Supported Hadoop Distributions: Add Support for Amazon EMR v6.0.0	DAP-40354
	Datameer X now supports Amazon EMR v6.0.0.

Bug Fixes

1	Import/ Export: Add tooltips for long column names in Full Data and Export Wizard step 'Choose Sheet'	DAP-38627
	Datameer X now renders tooltips for long column names.	Bug fixed.

2	Import Job: Browse all data, and job details are opened in separate tabs	DAP-39613
	Datameer X stays in the same browser tab during navigation after running an import job.	Bug fixed.

3	Import Job: 'DelegateImportFormat' and 'SequenceFileType' create an empty configuration instead of using the provided one	DAP-39884
	Datameer X now validates for properly configured 'io.serializations' settings for custom sequence file formats.	Bug fixed.

4	Import/ Export: Error during an ImportJob creation	DAP-40148
	While creating a BigQuery-based ImportJob in a Google BigQuery connection, no 'IllegalArgumentException' error is shown in the 'Data Details' step anymore.	Bug fixed.

5	Workbook: Can not rename a source column from a SQL joined sheet	DAP-40158
	Renaming a source column of a SQL sheet doesn't lead to an error any more.	Bug fixed.

6	Import: Can't import an AVRO file via HDFS HA Connection	DAP-40172
	Importing AVRO files from HA HDFS connections now works.	Bug fixed.

7	Setup: Datameer X doesn't clean up temporary data	DAP-40242
	Datameer X does a clean-up of its temporary folder during a start-up as long it is configured under Datameer's installation directory, but keeps it untouched otherwise.	Bug fixed.

8	HiveServer2 Plug-in: Enable support of mid-column schema changes of an underlying Hive table (AVRO & Parquet)	DAP-40267
	Datameer's Hive plug-in is supporting mid-column schema changes for Parquet and AVRO storage formats now.	Bug fixed.

9	File Browser: Details of an error dialog box opens a new browser tab	DAP-40292
	Error details are now rendered in the current browser tab.	Bug fixed.

10	Import Job: An import job is still referenced in a workbook after deleted the datasource in the workbook	DAP-40321
	When deleting a datasource, the current workbook holds no further references to it anymore.	Bug fixed.

11	Workbook: Workbook job run fails with an SQL sheet	DAP-40327
	The 'deploy.mode' is passed in for SQL sheets correctly now and the job run doesn't fail any longer.	Bug fixed.

12	Administration: The "User can see every file and folder" permission grants 'View Full Results' without an explicit mention.	DAP-40366
	Using this permission does not grant access to view data, only to see the files and folders. The ability to grant view full results became be a separate option to modify underneath.	Bug fixed.

13	Connection: Issue connecting to Amazon Athena	DAP-40372
	The connection to Athena now works when in the custom properties the 'das.jdbc.import.transaction-isolation=TRANSACTION_NONE' property is set.	Bug fixed.

14	File Browser: Abort in 'Duplicate Folder' or 'Paste Folder' does not work	DAP-40374
	Clicking the 'Abort' button while duplicating or pasting a folder now cancels the operation.	Bug fixed.

15	REST API: REST API v1 vs v2 - inconsistent handling of workbook UUIDs	DAP-40400
	The REST API v2 will generate a new random UUID during POST (while a valid UUID is required in the payload, e.g. '00000000-0000-0000-0000-000000000000' can be used as a template).	Bug fixed.

16	Plug-ins: Errors are swallowed, making it look like the plug-in was successfully initialized	DAP-40406
	A plug-in will not get registered if there has been an error during plug-in initialization.	Bug fixed.

17	HiveServer2: A workbook that contains a datalink drops the records after a schema change of the underlying Parquet table	DAP-40448
	Datameer X now imports all records from before and after a schema change.	Bug fixed.

18	Job Status Notification: Validation of the e-mail address according to RFC2822 (in detail: e-mail can contain "&")	DAP-40453
	Job status notifications for import, export and workbooks jobs can be send via e-mail addresses that contain chars like '&' from now on.	Bug fixed.

19	Support Engineer Report: The property 'SupportEngineerReportController.filterFilesLastNDays()' must cut at midnight from the system default timezone, not UTC	DAP-40467
	The 'Support Engineer Report' now respects the system default timezone.	Bug fixed.

20	Plug-in Tez: 'CONSUMED_BYTES/RECORDS' is incorrect in some cases	DAP-40471
	The plug-in now reports the counters for all (aliased) inputs in the dataset.	Bug fixed.

21	Plug-in Tez: A 'job-conf.xml' debug artifact should be written only once per Tez job, not for each vertex	DAP-40475
	The 'job-conf.xml' debug artifact is only written once for a job now, therefore rate limits like in GCP/ GCS don't occur.	Bug fixed.

...

Section

Datameer X 10.1.3.1

New Features

1	Supported Hadoop Distributions: Add Support for Cloudera CDP / Cloudera Runtime 7.0(.3)	DAP-39597
	Cloudera Runtime 7.0(.3) is now supported.

Improvements

1	Connection: Step 'Save' in connection edit mode - show a list of impacted artifacts when the authentication mode is set to 'SESSION_BASED'	DAP-39087
	A list of the impacted artifacts is now displayed when the authentication mode is set to 'SESSION_BASED' in the section 'Save' during connections.

2	Storybook: Enable new authentication dialog to request credentials	DAP-39088
	In the storybook a new React component is enabled which requests either user 'Name', 'Password' or 'SSH Key' depending on the used connection.

3	Supported Hadoop Distributions: Add support for Hortonworks HDP 3.1.5	DAP-39819
	Datameer X now supports Hortonworks HDP 3.1.5.

4	Tableau Export Connection: Parallel TDE/ Hyper file generation for Tableau Server exports	DAP-39846
	Setting the property 'das.splitting.disable-combining=true' while configuring a Tableau connection runs the jobs significantly faster. It is verified that an export of a sheet to the Tableau server launches multiple tasks.

5	Supported Hadoop Distributions: Add support for Cloudera CDH 6.3.3	DAP-39933
	Datameer X now supports Cloudera CDH 6.3.3.

6

Import job/ DataLink Wizard: Request credentials while edit/ create

DAP-39935

The property 'enable.authenticationOnJobRun=true' activates the session base authentication feature.

If a connection is configured as 'SESSION_BASED' and the authentication is already entered, the wizard will not show the ask for the credentials group.

If a connection switches from 'PROVIDED/TEMPORARY' credentials to 'SESSION BASED' and edit/ save the import job, a check after saving in the database table 'job_configuration_property' the properties 'datastore.user_name' and 'datastore.password' must be deleted/ not exist for a session based connection import/datalink job. Furthermore a check after saving in the database table 'dap_job_configuration' column 'schedule_enabled' must be true (1) for 'PROVIDED' and 'TEMPORARY' and false (0) for 'SESSION BASED'.

7	Workbook: Display a "Save" button in toolbar	DAP-40047
	You can now click on a 'Save' button in the UI toolbar again to safe changes within a workbook.

8	Workbook: Workbook Saving - Default to working path	DAP-40055
	When closing a workbook for the first time, the workbook is saved in the folder it was created from and not in home folder upon exit. A workbook message does not read "saved under home folder" but using the actual path where the workbook is stored. When closing the File Browser the workbook is saved in the folder it was created from.

9	Supported Hadoop Distributions: Add HBase 2 support for Cloudera CDH 6 versions	DAP-40109
	Datameer X now supports HBase 2 support for Cloudera CDH 6 versions.

10	Tableau Plug-in: Tableau Plug-in allow TDE/ Hyper exports to be setup from non-Linux systems	DAP-40147
	The plug-in for Tableau now allows TDE/ Hyper exports to be setup from non-Linux systems.

11	Workbook Settings: Disable scheduling if "request credentials on job trigger" condition is fulfilled	DAP-39939
	The scheduling menu is disabled if "request credentials on job trigger" conditions are fulfilled. A warning dialog is displayed why the 'Trigger' section is disabled. The REST v2 artifacts for job configurations now contain a new state if scheduling is enabled/ disabled.

12	REST API: Enable back authentication for REST API call via session cookies	DAP-40012
	Datameer X has two new security models for for REST API calls via session cookies which can be switched with the property 'enable.rest.csrf.checking=true/false'. The standard mode requires a CSRF token while the loose mode allows the authentication without a CSRF token.

Bug Fixes

1	Column Inspector: 'Undefined' shown as a formula string in case of empty columns	DAP-39529
	Empty columns do not show formulas in the column inspector any more.	Bug fixed.

2	Tableau Export: Tableau ExportJobs performance issue	DAP-40034
	The property 'das.splitting.disable-combining=true' while configuring a Tableau connection runs the jobs significantly faster and multiple tasks can be launched.	Bug fixed.

3	Workbook: Duplicating or copy/ pasting a workbook returns an error	DAP-40067
	Workbooks can be copied and pasted without an error again.	Bug fixed.

4	Workbook: 'EXPAND' function column position shows incorrect result for 'GROUPROWNUMBER'	DAP-40087
	Using the 'EXPAND(LIST)' function shows the correct sequence for 'GROUPROWNUMBER'.	Bug fixed.

5	Display key columns first flag causes 'IllegalStateException' in a 'Join Sheet'	DAP-40121
	Displaying the key columns first flag doesn't cause exceptions in a 'Join Sheet' any more.	Bug fixed.

6	Workbook: Can't rename a column in the 'Join Sheet' because of a 'RegularRequirementNotMetFailure'	DAP-40130
	Column renaming in a 'Joint Sheet' is possible because the broken references check implemented for transformed sheets doesn't affect renaming at other sheet types.	Bug fixed.

7	Upgrading Datameer X 7.5.x to 10.1.2 workbook upgrade conversion breakage	DAP-40140
	Column changes are propagated to all depended workbooks and sheet, not only when upgrading.	Bug fixed.

8	Avro Import: 'NoSuchMethodError' with Cloudera CDH-5.14.4 (Hadoop distribution jackson jar binary incompatible)	DAP-40142
	Importing Avro files works on Cloudera CDH-5.14.4 distributions.	Bug fixed.

9	Workbook: Datameer X OOM crash when viewing sorted sheet data	DAP-40151
	The new property 'das.data-download.merge-sort.max-total-file-size' is introduced to control (volume based) if the sorting will happen in full data view or not. The default value is 256MB.	Bug fixed.

10	Datameer X Mode: Prevent mixed content issues if Datameer X is configured in SSL mode	DAP-40176
	The button "New System" within 'External Systems' is available for entitled users such as an Admin in a SSL enabled installation, too.	Bug fixed.

11	Upgrading Process: After upgrade the search index could be wrong or missing	DAP-39528
	After the upgrading process, the old search index is deleted and a new index created.	Bug fixed.

12

Workbook: Pivot Sheet - While creating/ editing a Pivot sheet the buttons 'Toggle Column' and 'Go To Line' are not disabled

DAP-39770

When a Pivot sheet is created or an existing Pivot sheet is edited in a workbook, the button 'Toggle Column' is disabled.

When a Pivot sheet is created or edited in a workbook, the button 'Go To Line' is disabled.

On Pivot sheet while being in Pivot preview mode clicking on 'Split', 'De-Duplicate' or 'Encode Column', the toolbar buttons dismiss the Pivot review mode.

Bug fixed.

13	Workbook: 'Show Dependencies' is very slow and keeps loading	DAP-40023
	The 'Dependencies' are now displayed in an appropriate amount of time when clicking the icon.	Bug fixed.

14	Workbook: It is not possible to use a column in a join if the column was in an error state	DAP-40052
	You can use a column in a join if the column was in an error state before and switch sheet was involved.	Bug fixed.

15	Datameer X Grid Mode: EMR - Clustername nor ID are validated while saving the grid mode.	DAP-40100
	While saving an EMR connection configuration, the clustername and ID are now validated.	Bug fixed.

16	Neebo to Datameer X Accessibility: Data Warehouse plug-in shouldn't restrict the access to file descriptors by 'HADOOP_CLUSTER_ACCESS' capability	DAP-40201
	Datameer X data is accessible via Neebo since a user needs 'IMPORTJOB_DOWNLOAD' and 'WORKBOOK_DOWNLOAD' capabilities are guaranteed with at least the role 'ANALYST'.	Bug fixed.

17	Export Job: Options for 'On a schedule' are missing	DAP-40227
	When creating an export job, for the schedule method "on a schedule" several options are available, e.g. hourly, monthly.	Bug fixed.

Section

Datameer X 10.1.2

New Features

1	Add REST API call to look up what a UUID's properties are.	DAP-31229
	For a UUID Datameer X provides the JSON containing the information about the artifact analog to a REST READ call for an artifact ID.

Improvements

1	Connection: In wizard section 'details' validate and store authentication on a session if the authentication session-based option is selected	DAP-39334
	Authentication data is now validated and stored per session in the 'Connection Details' section within the connection wizard.

2	Connection: Show selected authentication mode on 'Connection Details'	DAP-39335
	Datameer X now shows the authentication mode on the 'Connection Details' page if the connection type provides such options (JDBC/MySQL, SSH, SFTP connections).

3	Connection: Implement a controller for session-based connection authentication	DAP-39936
	A new service checks the session id and connection id for session-based authentication connections.

Bug Fixes

1	Workbook: Inconsistency in error messages for workbooks without a shared source	DAP-38108
	Error messages are adjusted and are now consistent for errors when working with a shared workbook without a shared source.	Bug fixed.

2	Mysql 8.x execute mysql-init.sql fails - no Grant options allowed	DAP-39867
	Datameer X has two new MySQL scripts for running MySQL 5.x and MySQL 8.x databases.	Bug fixed.

3	Security: After a failed login attempt, Datameer X exposes the LDAP query to fetch users	DAP-39953
	The error message is shown again as expected and does not expose the LDAP query to fetch users anymore.	Bug fixed.

4	Google Cloud Storage is bundled with the wrong Hadoop library	DAP-39979
	The Google Cloud Storage connector is now bundled with all Hadoop 3 versions.	Bug fixed.

5	Infographics: DataSetsController REST API copies full workbook or import job data to DFS (EhCache), therefore the conductor disk space is filling up	DAP-39990
	The DFSCache button on the developer's page is working again and data is now cleaned up as expected.	Bug fixed.

6	DevController: Cache Monitor 'Clear Cache' doesn't purge data	DAP-40005
	The DFS cache is now cleared when clicking the "Clear Cache" button.	Bug fixed.

7	Workbook: Filtering on a column with the 'DENULLIFY' function causes an 'IllegalArgumentException'	DAP-40017
	The Datameer X function 'DENULLIFY' works as expected again.	Bug fixed.

8	FileBrowser: Increase polling interval for thr file browser's dependency view	DAP-40038
	Due to performance, the polling interval while working with the file browser's dependencies view is now increased to 100 sec.	Bug fixed.

9	Workbook: Typo in backup and restore error message	DAP-40046
	The error messages for duplicating and copying/pasting a shared workbook without sharing the data source is now correct.	Bug fixed.

...

Page Comparison

Versions Compared

Old Version 36

New Version Current

Key

Datameer X 10.1.16

Improvements

Datameer X 10.1.23

Bug Fixes

Datameer X 10.1.22

Bug Fixes

Datameer X 10.1.21

Improvements

Bug Fixes

Datameer X 10.1.20

Improvements

Bug Fixes

Datameer X 10.1.19

Bug Fixes

Datameer X 10.1.18

Bug Fixes

Datameer X 10.1.17

New Features

Improvements

Bug Fixes

Datameer X 10.1.16

Improvements

Bug Fixes

Datameer X 10.1.15

Improvements

Bug Fixes

Datameer X 10.1.14

New Features

Improvements

Bug Fixes

Datameer X 10.1.13

New Features

Improvements

Bug Fixes

Datameer X 10.1.12.2

Bug Fixes

Datameer X 10.1.12

New Features

Improvements

Bug Fixes

Datameer X 10.1.11

New Features

Improvements

Bug fixes

Datameer X 10.1.10

New Features

Improvements

Bug fixes

Datameer X 10.1.9

Improvements

Bug Fixes

Datameer X 10.1.8

New Features

Improvements

Bug Fixes

Datameer X 10.1.7

New Features

Improvements

Bug Fixes

Datameer X 10.1.6

New Features

Improvements

Bug Fixes

Datameer X 10.1.5

New Features

Improvements

Bug Fixes

Datameer X 10.1.4.1

Improvements

Bug Fixes

Datameer X 10.1.4

Improvements

Bug Fixes

Datameer X 10.1.3.1

New Features