INFO
Within these release notes Datameer X provides information and resolutions on improvements, bug fixes, and visual changes for 10.1.x maintenance release versions.
Visit Datameer's Help Center for additional information or contact your customer service representative.
Datameer X 10.1.19
Bug Fixes
1 | Security: 'SSLHandshakeException' - Protocol is not appropriate | DAP-42262 |
---|---|---|
Creating a SSL-secured IMAP connection can now be established after updating the libraries. | Bug fixed. |
Datameer X 10.1.18
Bug Fixes
1 | Execution Engine: Remove Spark execution engine from the Job Graph | DAP-42202 |
---|---|---|
Spark has been removed from the list of available execution frameworks from the Job Details page's Job Graph section. The Job Graph now shows only the available execution frameworks job durations. | Bug fixed. |
Datameer X 10.1.17
New Features
1 | Admin: Job Scheduler - Allow to bypass the Job Scheduler queue | DAP-41875 |
---|---|---|
If needed, you can now execute priorized import and export jobs by bypassing the Job Scheduler's queue. For that, according role capabilities are needed as well as cluster resources. |
Improvements
1 | REST API: Provide REST API commands to pause and resume the Job Scheduler and auto compaction | DAP-36390 |
---|---|---|
You can pause and resume both the Job Scheduler and auto compaction via REST API commands. |
Bug Fixes
1 | Import: Google BigQuery - Datameer X can not convert Google BigQuery DATE to Datameer X DATE format | DAP-42220 |
---|---|---|
When importing data from Google BigQuery to Datameer X, the correct pattern is used to convert from Google BigQuery DATE format to Datameer X DATE format. | Bug fixed. |
2 | Workbook: Display the autocomplete list for the current sheet and column name | DAP-42180 |
---|---|---|
When using a formula that contains the current sheet name and the column name, the autocomplete list is now displayed. | Bug fixed. |
3 | Connection: Twitter REST API - Can't get token, improve error message | DAP-42108 |
---|---|---|
If a user tries to configure a Twitter Rest API and pressing the 'Authorize Datameer to retrieve data' button results in an error, now the actual error message is shown. | Bug fixed. |
4 | Workbook: 'Close Workbook' dialog appears every second time when closing an existing Workbook with no changes | DAP-42044 |
---|---|---|
When closing an open Workbook, in which no changes were applied, the 'Close Workbook' dialog is not displayed at no time. | Bug fixed. |
5 | Import/ Export: PowerBI - Creating a connection and export job fails | DAP-41634 |
---|---|---|
A user can now create a connection to PowerBI again. Configuring and running export jobs is possible again as well. | Bug fixed. |
6 | REST API: Curl command for uploading a license should not create a user session | DAP-41581 |
---|---|---|
Using the curl command for uploading a license does not increase the session count for user sessions any more. | Bug fixed. |
7 | REST API: Curl command for uploading a license is not working | DAP-41356 |
---|---|---|
The curl command for uploading a license is working as expected again. | Bug fixed. |
Datameer X 10.1.16
Improvements
1 | Supported Hadoop Distributions: Add support for Amazon EMR v6.3.0 | DAP-42063 |
---|---|---|
Datameer X now supports Amazon EMR version 6.3.0. |
2 | Supported Hadoop Distributions: Add support for Amazon EMR v5.33.0 | DAP-42064 |
---|---|---|
Datameer X now supports Amazon EMR version 5.33.0. |
3 | Import: Oracle Eloqua - Importing Data into Datameer X | DAP-42077 |
---|---|---|
You can now ingest import data from Oracle Eloqua into Datameer X to receive campaign responses. For that, a JDBC driver is needed. |
Bug Fixes
1 | Workbook: Repoiting - 'NullPointerException' if the column to be repointed is blank | DAP-42141 |
---|---|---|
When repointing columns from a source sheet to a duplicated sheet, there is no 'NullPointerException' error anymore. | Bug fixed. |
2 | Upgrading Datameer X: Upgrade to Datameer X version 10.1.9 is blocked | DAP-42131 |
---|---|---|
After fixing a swallowed SQL sheet and fixing some minor dependencies, the upgrade could be executed successfully. | Bug fixed. |
3 | Workbook: Autocomplete in the 'Formula Builder' offers columns from all sheets | DAP-42105 |
---|---|---|
When inserting a formula in the Workbook's 'Formula Builder' only the sheets and columns for the first reference and only the columns from the current sheet are offered for autocompletion. Furthermore, autocomplete filters out dummy columns, e.g. insert a formula in the second column, apply, and then go to the first column. | Bug fixed. |
4 | Import/ Export: Databricks connection has the Export option under the Connection usage while it is not supported | DAP-42073 |
---|---|---|
The Connection usage list for a Databricks connection now contains only the connection type that is supported by the chosen connection type: Import. | Bug fixed. |
5 | Workbook: Renaming a column with a long name leads to a shift between the column header and the column | DAP-41921 |
---|---|---|
After renaming a column name, no shift is visible between the column header and the column. | Bug fixed. |
Datameer X 10.1.15
Improvements
1 | Workbook: Provide an option to cancel the 'Repoint Sheet' dialog. | DAP-42003 |
---|---|---|
You can now cancel the 'Repoint Sheet' dialog after first applying a formula and then saving the sheet name. The applied formula is also being reset after cancelling the dialog. |
Bug Fixes
1 | Workbook: Workbook still appears in the tab bar after closing the Workbook | DAP-42027 |
---|---|---|
After closing a Workbook, the associated Workbook tab is being closed in the tab bar as well. | Bug fixed. |
2 | Import: JSON import - Datameer ignores the first array object | DAP-41996 |
---|---|---|
All arrays of the JSON file are being imported. With the 'include root array' property, Datameer parses the whole array including the first object also. In general when importing JSON: If a JSON file contains a 'root array', means the first level if the data structure is a JSON array (file starts with [...]), the 'include array' checkbox has to be checked during the import wizard. If a JSON file contains an object in the first level and a nested data structure contains an array for the data ingest (file starts with {...[...]...}), the 'include root array' must not be checked. Then the JSON path to the 'target array' for the data ingest must be entered into the textbook during the import wizard. | Bug fixed. |
3 | Datameer Distributions: Azure Blob Storage Connector - Packaging issues | DAP-41979 |
---|---|---|
After updating the core libraries, the Azure Blob Storage connector is working as expected again. | Bug fixed. |
4 | In-product documentation: Wrong documentation link for 'New Connection' | DAP-41944 |
---|---|---|
A user who wants to find out about more information about the 'Data Types' during the 'New Connection' workflow, is now directed to the correct public documentation page. | Bug fixed. |
Datameer X 10.1.14
New Features
1 | Supported Hadoop Distributions: Add support for Google Dataproc version 1.5 | DAP-40353 |
---|---|---|
Datameer X now supports Google Dataproc version 1.5. |
Improvements
1 | Supported Hadoop Distributions: Add support for Cloudera Private Cloud/ Cloudera Runtime 7.1.6 | DAP-41823 |
---|---|---|
Datameer X now supports Cloudera Private Cloud/ Cloudera Runtime 7.1.6. |
2 | Admin Plug-ins: ‘plugin-tez’ - Upgrade to TEZ 0.10 | DAP-41279 |
---|---|---|
After a plug-in patch, the 'plugin-tez' plug-in is now available for Datameer X distributions again. |
3 | Properties: AWS S3 http client max pool size should be configurable | DAP-41912 |
---|---|---|
The default maximum connection execution amount is now set to 100. With custom properties, you can change this amount to a higher or lower number. The custom count can also be set via Hadoop property in the cluster settings. |
4 | Workbook: Create Toolbar button to auto-expand all column width to make all column names visible | DAP-41603 |
---|---|---|
Datameer has now a new toolbar button to expand all column width to make all column names fully visible. |
5 | Workbook: Formula sheet repointing | DAP-41549 |
---|---|---|
When changing a formula sheet that points to another sheet, Spectrum now detects this changes and displays a dialog: The user can apply the changes to only the current column or to all columns. This dialog is only displayed, when when more than one column is affected. |
Bug Fixes
1 | Workbook: Parquet format - Ingesting creates duplicate 'das' metadata columns | DAP-41874 |
---|---|---|
When ingesting a Parquet file into Datameer by a File Upload, Import Job or Datalink, there will be no more duplicated columns displayed in the tab "Define Fields". | Bug fixed. |
2 | Snowflake Export: Tables are not updating when selecting another schema | DAP-41721 |
---|---|---|
When configuring a Snowflake export or import job, the tables are now updated when changing the schema from the drop-down. | Bug fixed. |
3 | Workbook synchronisation: Sheet synchronization - Validation error is still showing even after update | DAP-41476 |
---|---|---|
After a formerly incorrect formula has been changed into a valid formula, the validation error is no longer displayed. | Bug fixed. |
4 | FileBrowser: Remove the 'Import Job' tab from the tab bar after the job has been deleted | DAP-41635 |
---|---|---|
The 'Import Job' tab is now deleted from the tab bar and is no longer displayed, after the job has been deleted. | Bug fixed. |
Datameer X 10.1.13
New Features
1 | Export: Amazon Redshift incremental export | DAP-35849 |
---|---|---|
Incremental export to Amazon Redshift is now available when configuring an export. For that, select "Incremental" as the Record Replacement Policy and select the required column from the dataset. |
Improvements
1 | Supported Hadoop Distributions: Add support for Cloudera Private Cloud/ Cloudera Runtime 7.1.5.14 | DAP-41578 |
---|---|---|
Datameer X now supports Cloudera Private Cloud/ Cloudera Runtime 7.1.5.14. |
Bug Fixes
1 | Workbook: Workbook toolbar buttons are not working when using them in the Internet Explorer | DAP-41864 |
---|---|---|
The Datameer X Workbook toolbar now works with Internet Explorer. | Bug fixed. |
2 | Export: Amazon Redshift (Fast Load) export is dropping records | DAP-41838 |
---|---|---|
The data export into Amazon Redshift (Fast Load) is working correctly again. | Bug fixed. |
3 | Workbook: Workbook toolbar buttons are not working when using them in the latest Chrome version | DAP-41836 |
---|---|---|
The Datameer X Workbook toolbar now works with Chrome version 89.0.4389.90 (Official Build) and newer. | Bug fixed. |
4 | Execution Engine: Datameer X on Amazon EMR/ Google Cloud Platform - Failed to activate the plug-in 'plugin-dwh' | DAP-41816 |
---|---|---|
Since 'plugin-doh' is a conductor-only plug-in, it is not being copied to the cluster anymore and does not impact any job execution. | Bug fixed. |
5 | Import/ Export: Snowflake - Tables are not updating when selecting another schema | DAP-41721 |
---|---|---|
When configuring a Snowflake export or import job, the tables are now updated when changing the schema from the drop-down. | Bug fixed. |
6 | User Interface Login: Intermittently login page is not refreshing after a successful login | DAP-41633 |
---|---|---|
The correct login is working in all browsers for SAML, internal authentication as well as for external authentication (LDAP). | Bug fixed. |
Datameer X 10.1.12.2
Bug Fixes
1 | Workbook: Workbook toolbar buttons are not working when using them in the Internet Explorer | DAP-41864 |
---|---|---|
The Datameer X Workbook toolbar now works with Internet Explorer. | Bug fixed. |
2 | Workbook: Workbook toolbar buttons are not working when using them in the latest Chrome version | DAP-41836 |
---|---|---|
The Datameer X Workbook toolbar now works with Chrome version 89.0.4389.90 (Official Build) and newer. | Bug fixed. |
Datameer X 10.1.12
New Features
1 | Supported Hadoop Distributions: Add support for Amazon EMR 6.2.0 | DAP-41575 |
---|---|---|
Datameer X now supports Amazon EMR 6.2.0. |
2 | Supported Hadoop Distributions: Add support for Amazon EMR 5.32.0 | DAP-41574 |
---|---|---|
Datameer X now supports Amazon EMR 5.32.0. |
Improvements
1 | Workbook: Auto-expand column header during "column edit" if the column name is partly invisible | DAP-41677 |
---|---|---|
When renaming a column, the column width is now expanded fully visible without any truncation. |
2 | Supported Hadoop Distributions: Enable Apache Hadoop 2.8.5 support | DAP-41451 |
---|---|---|
Datameer X now supports Apache Hadoop 2.8.5, also with Hive 2.3.8 as well as HBase 1.4.13. |
Bug Fixes
1 | Export: Snowflake - Export jobs failing with exception | DAP-41557 |
---|---|---|
Snowflake export jobs run successfully again after fixing the plug-in. | Bug fixed. |
2 | File Browser: Dependencies View artifact loses its color after selecting/ deselecting | DAP-41322 |
---|---|---|
In the Dependencies View a selected and then deselected artifact does not change the color to grey anymore. | Bug fixed. |
Datameer X 10.1.11
New Features
1 | Datameer Service: New Event Logging Plug-in | DAP-41323 |
---|---|---|
The new Event Logging plug-in allows to log an artifact creation or deletion event. Certain types can be enabled and disabled for logging them. If an event type is not checked, no entry will be logged if the action happens in Datameer X. The log file is then downloadable with the System and Usage Report. |
Improvements
1 | Workbook: Auto-expand column header during "Column edit" if the column name is partly invisible | DAP-41677 |
---|---|---|
When renaming a column, the column width is now expanded fully visible without any truncation. |
2 | Supported Hadoop Distributions: Add Support for Cloudera CDP-DC/ Cloudera Runtime 7.1.4.1 | DAP-41067 |
---|---|---|
Datameer X now supports Cloudera CDP-DC/ Cloudera Runtime 7.1.4.1. |
Bug fixes
1 | Export: Snowflake - Invalid 'DATE' type export | DAP-41329 |
---|---|---|
The Datameer 'DATE' values are displayed properly as Snowflake's 'TIMESTAMP' values or other data-like data types when exporting. | Bug fixed. |
2 | Performance: JDBC-based Data Link very large sample jobs do not limit the records, read and run too long on large tables | DAP-41334 |
---|---|---|
Due to performance improvements, JDBC-based Data Link sample jobs finish faster and collect 5,000 sample records by default. | Bug fixed. |
3 | Service: Conductor.sh Check triggers tmp/ directory clean up | DAP-41336 |
---|---|---|
The tmp/ directory clean up is not triggered while Datameer X is running any more. The Conductor.sh Check command does not trigger the tmp/ directory clean up. | Bug fixed. |
4 | Workbook: Pivot Sheet throws and NullPointerException in case its source sheet has no records | DAP-41620 |
---|---|---|
When a source of a Pivot sheet has no records, an error is now displayed in the Pivot sheet that there is no data in the source sheet. | Bug fixed. |
5 | Connections: Amazon S3 - IAM import/ export failure | DAP-41667 |
---|---|---|
After replacing the jar files the import and export are working as expected again. | Bug fixed. |
6 | File Browser: Dependencies View artifact loses its color after selecting/ deselecting | DAP-41322 |
---|---|---|
In the Dependencies View a selected and then deselected artifact does not change the color to grey anymore. | Bug fixed. |
7 | Workbook: SQL sheets - TRIM function in WHERE clause | DAP-41380 |
---|---|---|
After applying the LIKE clause the TRIM function returns the required records accordingly. | Bug fixed. |
Datameer X 10.1.10
New Features
1 | Import/Export: Google BigQuery - Add partition filtering functionality | DAP-41202 |
---|---|---|
It is now possible to import from a partitioned table. |
2 | Workbook: Custom functions for dictionary matching | DAP-41237 |
---|---|---|
Datameer X has now two new functions for dictionary matching: "MATCHING_DICTIONARY" and "MATCHING_DICTIONARY_FUZZY". |
3 | Import/ Export: S3 connection - Add the IAM role ARN option | DAP-41298 |
---|---|---|
There is a new authentication option for the S3 connection: You can now authenticate via the IAM role with an assumed role. |
Improvements
1 | Plug-Ins: plugin-hive - do not overwrite the fetch size and use the value defined in the jdbc connection string | DAP-41130 |
---|---|---|
The fetch size is new increased up to 5,000 to boost the jdbc-based import jobs. |
2 | Authentication: Terminate all user sessions on logout | DAP-41159 |
---|---|---|
Datameer X presents a new property 'devicesconductor.session.management.enabled'. If it is set to true, a second login with the same user will be prohibited. The property is disabled by default. |
3 | Admin UI: Database driver tab - wide grid or horizontal scrollbar | DAP-41197 |
---|---|---|
The UI has now more screen space for the grid, showing the installed JDBC drivers. Resizing the File Browser window, the width of the table is being adjusted. |
4 | Export: Hive - Improve the help text for the replacement strategy during the Hive export | DAP-41223 |
---|---|---|
The user now sees a more specific help text for the replacement strategy for a Hive export. |
Bug fixes
1 | File Browser: Disable the folder context menu 'Rename' in the Folder Tree | DAP-40713 |
---|---|---|
If no folder is selected within the Folder Tree the 'Rename' option from the context menu will be disabled. | Bug fixed. |
2 | File Browser: Wrong documentation link for the file search | DAP-40889 |
---|---|---|
When searching for files in the File Browser the help link now points to the correct corresponding documentation page. | Bug fixed. |
3 | Workbook: Wrong color highlighting of formula parameter in the Formula Editor | DAP-40933 |
---|---|---|
Formula highlighting works as expected again. | Bug fixed. |
4 | Workbook: Scheduling information is shown on a Workbook Details page even when the scheduling is disabled | DAP-41097 |
---|---|---|
The scheduling information from the Workbook Details page is now up-to-date after changing the scheduling manually. | Bug fixed. |
5 | Datameer Distribution: Jars missing in installation package for CDH-7.x.x | DAP-41180 |
---|---|---|
The installation package for Cloudera CDH-7.x.x distributions is now complete. | Bug fixed. |
6 | Connection: 'Save Connection' dialog - 'Close' is visible under the 'close' icon | DAP-41205 |
---|---|---|
The string 'close' within the 'Save Connection' dialog does not appear anymore. | Bug fixed. |
7 | Workbook: Post Upgrade Issue - NullPointerException in partition filter with $lastestpartition | DAP-41222 |
---|---|---|
Datameer X will not throw a NullPointerException anymore if a user tries to update the partition filter in a Workbook. | Bug fixed. |
8 | REST API: Concurrent session management blocks REST API to work | DAP-41242 |
---|---|---|
The concurrent session feature still works when enabled. The REST API works independently from enabling or disabling the concurrent session feature. | Bug fixed. |
9 | File Browser: Dependencies View only opens after the second attempt | DAP-41291 |
---|---|---|
Clicking the 'Dependencies View' icon now opens the Dependencies View at the first try. | Bug fixed. |
10 | Workbook: Attempt to duplicate the Workbook leads to broken references in the new artifact | DAP-41297 |
---|---|---|
Duplicating or copying an artifact does not lead to broken references in the new artifact anymore. | Bug fixed. |
Datameer X 10.1.9
Improvements
1 | Workbook Health Check: Log status and progress into separate logs file 'logs/workbook_health.log' | DAP-41120 |
---|---|---|
The Workbook Health Check logs are now being written into the file 'logs/workbook_health_check.log'. |
2 | Supported Hadoop Distributions: Add support for Cloudera CDH 6.3.4 | DAP-41138 |
---|---|---|
Datameer X now supports Cloudera CDH 6.3.4. |
3 | Plug-ins: 'Plugin-dwh' - Make Data Warehouse plug-in default for Datameer distribution | DAP-41150 |
---|---|---|
The plugin 'plugin-dwh' is now available as a default plugin and comes automatically with all Datameer X distributions. |
Bug Fixes
1 | File Browser: After canceling 'Create new folder', the File Browser does not update when selecting another folder | DAP-40673 |
---|---|---|
After canceling 'Create new folder', it is possible to change the folder in the Folder Tree panel. | Bug fixed. |
2 | Pivot Sheet: Creating a Pivot sheet with a broken Workbook source ends up in an error dialog | DAP-40729 |
---|---|---|
Creating a Pivot sheet out of a broken Workbook is not possible since the error state of a broken Workbook is shown first before one can create a sheet out of a broken Workbook. | Bug fixed. |
3 | Workbook Details: The table for SheetDependent is missing/ not displayed | DAP-40821 |
---|---|---|
The table for 'SheetDependent' ("Consumed by x artifacts") is visible on the Workbook Details page. | Bug fixed. |
4 | File Browser: Prevent the option to open the 'Job run details' page if the necessary capability is missing | DAP-41129 |
---|---|---|
If a user does not have the capability “Common job details“ he will not enter the job-execution details page and an error message is shown instead. | Bug fixed. |
5 | Import: Clickhouse - Ingest of empty tables is not working | DAP-41140 |
---|---|---|
Ingesting empty tables now works properly when pointing to an empty table in a Clickhouse database. | Bug fixed. |
Datameer X 10.1.8
New Features
1 | Import/ Export: Enable Import to ClickHouse via a JDBC connection | DAP-40951 |
---|---|---|
Importing data via a JDBC connection from a ClickHouse database is now possible. You need to have the ClickHouse database driver installed and then configure the JDBC connection to execute imports and exports. |
2 | Supported Hadoop Distributions: Add support for Amazon EMR 5.31.0 | DAP-41025 |
---|---|---|
Datameer now supports Amazon EMR version 5.31.0. |
3 | File Browser: Add new root folder in the Folder Tree panel | DAP-41033 |
---|---|---|
A user can now add a new top-level folder/ root folder in the Folder Tree panel in the File Browser. The newly created folder is named 'Untitled Folder' and can be renamed immediately after creating it. The capability 'CREATE_FOLDER' is needed. |
Improvements
1 | Workbook Health Check: Enable the download of the the report and print the configuration IDs of all broken Workbooks | DAP-40628 |
---|---|---|
You can now download the Workbook Health Check as a JSON from the Workbook Health Check page. Furthermore the configuration IDs of all broken Workbooks are displayed as a comma-separated list unter the Workbook Health section. |
2 | Rebranding: Datameer Classic is now DatameerX | DAP-40913 |
---|---|---|
In course of the rebranding, The former 'Datameer Classic' is now called 'DatameerX'. The new Datameer products are called 'Spectrum' and 'Spotlight'. Find more information about all of our products on our website. |
3 | SFTP/ SCP import: Reduce the amount of the opened connections while multiple small file import | DAP-41011 |
---|---|---|
The new configuration property 'das.cluster.hadoop.fs.cache.scheme.whitelist' which is configurable globally and on job level, reduces the amount of SFTP/ SCP sessions. With the FileServer cache enabled for scp/sftp scheme, there will be a single SSH connection for each cluster task launched, limiting the "connection refused" occurances. |
4 | Authentication: SAML authentication - Disable the local account management | DAP-41058 |
---|---|---|
The mixed-mode parameter that defines whether or not local authentication is available while using SAML (or LDAP), can now be configured within the SAML plug-in settings. Having it set to false, only the external authentication mode is available. Having it set to true, both external and local authentication mode are available. |
5 | Connection: Amazon S3 Native connector should validate the region and bucket combination properly | DAP-41074 |
---|---|---|
When setting up an Amazon S3 Native connection the region is now validated against the bucket properly. |
6 | Housekeeping Service: Housekeeping Service should attempt to delete HDFS data artifacts for non-existent users | DAP-41090 |
---|---|---|
Deleting artifacts that are owned by a removed Datameer user is now possible. Deleting can be executed a Datameer Service user that has direct access to the HDFS cluster. |
Bug Fixes
1 | User Interface: Scrolling problems when the browser is vertically maximized | DAP-40723 |
---|---|---|
When the browser window is vertically maximized, the scroll bar works as expected and you can scroll up and down within a Workbook. | Bug fixed. |
2 | Sequential triggering: Do not display a complete list of all variables on the import/ export job run details page | DAP-41007 |
---|---|---|
The list 'Variables' from the import/ export job run details page contains only the used variables. | Bug fixed. |
3 | File Browser: Unable to 'Duplicate Workbook', the dialog remembers the previous filename value | DAP-41020 |
---|---|---|
Duplicating a Workbook works as expected. Executing the duplication creates a copy of the file and can be renamed as required. The default name of a duplicated Workbook is 'Copy_of_<initial Workbook name>'. | Bug fixed. |
4 | Connection: Google BigQuery export fails with staging bucket name that contains underscores | DAP-41045 |
---|---|---|
It is now possible to have the Google Cloud Storage bucket containing underscores as the Google BigQuery connector staging bucket/ directory. | Bug fixed. |
5 | Login: After session timeout - error when first re-login attempt | DAP-41054 |
---|---|---|
A user is able to log in to the Datameer User Interface after a session timeout from the first attempt on. | Bug fixed. |
6 | Security: Datameer Support Engineer Report extension allows reading random files from the Datameer application server OS | DAP-41059 |
---|---|---|
The Datameer extension Support Engineer Report reads files from the Datameer installation folder only (configuration as well as log files). | Bug fixed. |
7 | REST API: Path information disclosure when calling for a non-existent file | DAP-41060 |
---|---|---|
When calling for a non-existent file a '404' error is shown containing the requested file name. | Bug fixed. |
8 | Security: 'productDASData' text/ javascript block is available at the page's source even when the 'About' menu is disabled | DAP-41061 |
---|---|---|
When the 'About Datameer' menu is disabled, the 'productDASData' text/ javascript block is deactivated as well. | Bug fixed. |
9 | Plug-ins: plugin-s3 (S3 Native connector) on EMR - Export to a bucket in different S3 region fails | DAP-41073 |
---|---|---|
The plug-in is fixed. An export via the S3 Native connection works regardless of a bucket region. | Bug fixed. |
10 | Export: An export job should not inject 'DataStoreModel#setupConf' into the global 'JobConf' | DAP-41088 |
---|---|---|
An export job creates a dedicated 'JobConf' for export-only purposes. | Bug fixed. |
11 | Authentication: EMR- S3 connection uses IAM mode always when the cluster mode S3 is set to IAM | DAP-41100 |
---|---|---|
The Amazon S3 connection uses the configured authentication mode from the connection configuration: Access Secret/ Key. | Bug fixed. |
Datameer X 10.1.7
New Features
1 | Supported Hadoop Distributions: Add support for Amazon EMR 6.1.0 | DAP-40876 |
---|---|---|
Datameer X now supports Amazon EMR version 6.1.0. |
Improvements
1 | Import/ Export: Support for Azure Synapse (formerly Microsoft SQL Data Warehouse) | DAP-27739 |
---|---|---|
You can now import from or export to an Azure Synapse Database. The according dialect is being supported as well. |
2 | Supported Hadoop Distributions: Add support for Amazon EMR 5.30.1 | DAP-40875 |
---|---|---|
Datameer X now supports Amazon EMR version 5.30.1. |
3 | Supported Hadoop Distributions: Add support for Cloudera CDP-DC/ Cloudera Runtime 7.1.3 | DAP-40877 |
---|---|---|
Datameer X now supports Cloudera CDP-DC/ Cloudera Runtime version 7.1.3. |
4 | Connection: Rewrite Google BigQuery import connector to use Google BigQuery storage API | DAP-40883 |
---|---|---|
When importing from or exporting to Google BigQuery, the new Google BigQuery storage API is used for more efficient imports and exports.. |
5 | Authentication: Remove 'gssapi-with-mic' from Datameer's 'default.properties' | DAP-40799 |
---|---|---|
The value 'gssapi-with-mic' is removed from the 'default.properties' key 'system.property.jsch.preferred.authentication'. The new added value is 'password,keyboard-interactive,publickey'. |
6 | Authentication: SAML - Add option to read roles only from repository | DAP-40886 |
---|---|---|
You can now configure the SAML authentication to read all roles from the repository. Furthermore all four assertion properties are configurable (username, email, groups, roles). |
7 | Supported Hadoop Distributions: Add support for Apache Hadoop 2.7.7 and Hive 2.3.7 | DAP-40895 |
---|---|---|
Datameer X now supports Hadoop version 2.7.7 and Hive version 2.3.7. |
8 | Workbook: Add ability to 'update' instead of 'replace' for 'Cascade Permissions' | DAP-40916 |
---|---|---|
A user can now add permissions within 'Cascade Permissions' from a folder in the File Browser instead of fully replace the existing permissions. |
9 | Google BigQuery: Do not use temporary table when streaming from tables | DAP-40939 |
---|---|---|
Imports from Google BigQuery now work without changes for tables, views and queries. |
10 | Plug-ins: JDBC/Snowflake - Unimplemented date/ time/ timestamp column splitting | DAP-40952 |
---|---|---|
The plug-in 'plugin-jdbc-v2' is now adapted to the column splitting needs. |
11 | Supported Hadoop Distributions: Add support for Cloudera CDP-DC/ Cloudera Runtime 7.1.2 | DAP-40998 |
---|---|---|
Datameer X now supports Cloudera CDP-DC/ Cloudera Runtime version 7.1.2. |
12 | Supported Hadoop Distributions: Add support for Cloudera CDP-DC/ Cloudera Runtime 7.2.0.2 | DAP-40999 |
---|---|---|
Datameer X now supports Cloudera CDP-DC/ Cloudera Runtime version 7.2.0.2. |
Bug Fixes
1 | Workbook: SQL - 'EXCEPT' and 'INTERSECT' operator null value handling | DAP-40739 |
---|---|---|
The Datameer X functions 'EXCEPT' and 'INTERSECT' work properly again. The null value handling returns an empty result as expected. | Bug fixed. |
2 | Authentication: The default LDAP user filter does not filter out objects with username or email attribute set as an empty string | DAP-40884 |
---|---|---|
The corresponding plugin is patched. User objects with and username or email attribute set to an empty string are now filtered out and do not break the LDAP cache creation. | Bug fixed. |
3 | Workbook: Validate for sheet error when changing column type which is used for an encoding sheet | DAP-40892 |
---|---|---|
The sheet error is being registered and a proper error message is shown when changing the column type which is used for an encoding sheet. | Bug fixed. |
4 | Workbook: Inaccurate status and missing dialog if triggering the Workbook from configuration page | DAP-40915 |
---|---|---|
If triggering the Workbook from the Workbook Configuration page by the 'Starting calculation process immediately after save' option the process dialog is visible again and the status bar shows the accurate status. | Bug fixed. |
5 | Workbook: The key shortcut 'ALT + O' does not work | DAP-40923 |
---|---|---|
Executing the key shortcut 'ALT + O' opens the Workbook from the File Browser as expected. | Bug fixed. |
6 | Connection: ZooKeeper - Datameer X does not close ZooKeeper connections (HBase) | DAP-40970 |
---|---|---|
The HBase plug-in was fixed so Datameer X now closes ZooKeeper connections to HBase again when the connection is saved. | Bug fixed. |
7 | Connection: MariaDB dialect is missing export transformation 'SqlDateTransformation' | DAP-40989 |
---|---|---|
The needed export transformation is now added. MariaDB is changed to apply the export transformation 'SqLDateTransformation'. | Bug fixed. |
8 | Plug-ins: JDBC 2/ Snowflake - Integer column splitting skips table rows for import | DAP-40990 |
---|---|---|
All splitted columns from type integer can be imported completely. | Bug fixed. |
9 | Workbook: Sequential triggering does not respect the execution order | DAP-41000 |
---|---|---|
Sequential triggering executes the artifacts based on the execution order. | Bug fixed. |
Datameer X 10.1.6
New Features
1 | Admin: Support for token authentication within the Tableau plug-in | DAP-40342 |
---|---|---|
You can now select the authentication mode 'SSO Token' when creating a connection to a Tableau Server. |
2 | Workbook: New call to insert usage of sheets into Workbook config and details page | DAP-40722 |
---|---|---|
A new workbook query includes the usage of a sheet. The new call includes the information and displays it in the workbook configuration and workbook details page. Therefore a user knows if a sheet must be kept or not. |
Improvements
1 | Workbook Health Check: Provide the possibility to hide the loading screen and cancel the check | DAP-40607 |
---|---|---|
You can now hide the Workbook Health Check loading screen and continue working while the check is executed in the background. Furthermore you can cancel the check completely. |
2 | Security: SAML authentication - role mapping proceeding | DAP-40742 |
---|---|---|
In the 'Admin' tab in the SAML authentication you can now assign a role and/ or a group to a user directly, e.g. 'User1: ANALYST:sales'. Note that the group must exist when assigning the group to the role. |
3 | Logging: New log file to store Datameer X PID (Process ID) | DAP-40761 |
---|---|---|
A new log file named 'start-pid.log' is introduced to log the Datameer X start time and the corresponding process ID. Whenever Datameer X starts, a new line is being added to the log file. Find the log file under '/logs/dir'. |
4 | Workbook: Search field and 'go to column' feature in the Workbook Sheet Inspector | DAP-40338 |
---|---|---|
A user can now search for a column in the Workbook Sheet Inspector and also go directly to the column by clicking it. |
5 | Workbook: Show each column in error state separately/ more prominent | DAP-40370 |
---|---|---|
Columns in an error state are highlighted in an error color in the columns list inside the Workbook Sheet Inspector. Clicking the column in error state highlights the column in the sheet. Furthermore the column list height is resizable now. |
6 | Workbook: No consumer indication in the workbook configuration dialog if a sheet is configured to be synced via Open Document Format | DAP-40818 |
---|---|---|
The workbook configuration dialog now shows an indication that also synced sheets have a dependency consumed by. A workbook detail page contains truncated entries and a tooltip if the tables are longer than the rendered column width. |
7 | Security: Make SAML plug-in aware of a load balancer that offloads SSL | DAP-40868 |
---|---|---|
The load balancer offloads SSL to the Datameer X SAML plug-in. A new field is provided that enables - in addition to the existing Jetty-based SSL termination - the load balancer-based SSL offloading. |
Bug Fixes
1 | Export: Can't export strings values into MariaDB | DAP-38661 |
---|---|---|
New Hadoop configuration parameters, that may be set in the 'Admin' tab → 'Cluster Configuration' section or during the export job configuration, override the dialect's default varchar field length. The export to MariaDB is now possible. | Bug fixed. |
2 | Workbook: Error message isn't visible in the File Browser if trying to run an export-only workbook/ sheet without data | DAP-40440 |
---|---|---|
The error message that an export job of a workbook or sheet without data is failing, is now displayed again. | Bug fixed. |
3 | File Browser: 'Dependencies graph' performance | DAP-40727 |
---|---|---|
The performance for using the 'All Dependencies' and 'Depends On' function in the File Browser is now improved. | Bug fixed. |
4 | Workbook: Columns.unequalCount error in join sheet | DAP-40753 |
---|---|---|
An error is shown when submitting a join definition with duplicate keys also stating how to recover. Find more information here. | Bug fixed. |
5 | Workbook: IllegalArgumentException for join operation with multiple conditions | DAP-40754 |
---|---|---|
Join operations with multiple conditions work again. | Bug fixed. |
6 | Connections: Databricks import connector throws an 'org.apache.spark.sql.AnalysisException' | DAP-40764 |
---|---|---|
The exception is fixed and the import works as expected. | Bug fixed. |
7 | Admin: Non-admin user can change folder ownership and select other users with the same role | DAP-40767 |
---|---|---|
If a user has no capability 'Change Folder Owner' but the capability 'User can modify every file and folder' he is not able to change the owner of the folder. | Bug fixed. |
8 | Import: Columns of UUID type are being dropped from a PostgreSQL import | DAP-40794 |
---|---|---|
The UUID type is now supported for the import and therefore the import job is executed successfully. | Bug fixed. |
9 | Workbook: Can't duplicate or copy a workbook if the sheet execution order does not match the view order | DAP-40805 |
---|---|---|
Duplicating or copying a workbook works as expected and Datameer X follows the sheet logical order regardless of their current sequential position. | Bug fixed. |
10 | Database drivers: The MariaDB database driver has a wrong name in the JDBC connection string template | DAP-40855 |
---|---|---|
The MariaDB database JDBC connection string now has the correct string template and you can establish a connection as expected. | Bug fixed. |
11 | User Interface: Workbook icons are disabled for the own workbook | DAP-40871 |
---|---|---|
After the code was patched, the icons for a workbook are available and not greyed out anymore in the user interface. | Bug fixed. |
Datameer X 10.1.5
New Features
1 | Database Support: Create Azure Cosmos database dialect | DAP-40590 |
---|---|---|
You can now connect Datameer X against Azure Cosmos. Therefore you need to configure Azure Cosmos DB as a connector and install the corresponding JDBC driver. |
2 | Supported Hadoop Distributions: Add support for MapR Extension Package 6.3 with MapR 6.1 | DAP-40652 |
---|---|---|
Datameer X now supports MapR Extension Package 6.3 with MapR 6.1. |
Improvements
1 | Documentation: Help links shouldn't be shown in the UI when the property pointing to the documentation is invalid or empty | DAP-40386 |
---|---|---|
No links that are pointing to the Datameer X documentation are shown in the UI when the 'doc.url' property is not set. The 'Help' link in the context menu does not appear anymore. |
2 | Workbook: Add indicator in workbook configuration and workbook details page if a sheet is used as dependency | DAP-40444 |
---|---|---|
There are new indicators for sheets within a workbook whose sheets are either a source data or for an export. The indicator appears on both the workbook configuration page next to each sheet, as well as in the workbook's detail page. |
3 | Admin: Workbook Health Check improvements | DAP-40479 |
---|---|---|
You can now zoom in and out within the health check graph and navigate back to the original size. Furthermore you can view all broken workbooks in a expandable list within the workbook health report. The workbook IDs are clickable and point to the workbook node in the health graph. If a node is then selected it can be opened in the info panel. |
4 | Datameer X White Labeling: White labeling Datameer X for OEM | DAP-40550 |
---|---|---|
Several default properties were created to change the Datameer X UI and URL to have the Datameer X application white labeled. Furthermore the Datameer X distribution allows to change the small and big logo as well as favicons. |
5 | Performance: Optimize Browser Service WB#isSync calls for better performance | DAP-40565 |
---|---|---|
The workbook sync call is optimized to show the last sync and current sync status in the File Browser. |
6 | Workbook: Implement a workbook dependency cycle detector | DAP-40706 |
---|---|---|
A new workbook dependency cycle detector to identify workbook dependency cycles is implemented on the '/dev' page. After all workbooks are checked, a HTML report is accessible. A workbook is checked for being part of a workbook cycle. |
7 | Performance: Change task order in Housekeeping Service | DAP-40711 |
---|---|---|
A new default property 'housekeeping.run.task-attempts-per-run' is set to a value of '50' per default to decrease the performance of the Housekeeping Service significantly when it is to low. |
Bug Fixes
1 | FileBrowser: Could not delete a file upload after saving a copy | DAP-39776 |
---|---|---|
Deleting a file upload after saving a copy now works again. | Bug fixed. |
2 | Workbook: If a user has both "view" and "edit" permissions they should be able to rename a folder | DAP-40544 |
---|---|---|
A user which has the new capability "Rename Folder" and edit permissions to the parent folder, he can rename a child folder within a parent's folder. Having only the "Rename Folder" permissions but not the permissions to edit the parent folder, the user can rename the folders that are located in the parent's folder. A user of a folder can always rename the folder's default settings. | Bug fixed. |
3 | ImportJob: Change links in 'Import Job Details' view to actual buttons | DAP-40382 |
---|---|---|
The button 'Browse Data' works as expected again. | Bug fixed. |
4 | Import/ Export: Can't connect to the SFTP server hidden behind a proxy | DAP-40401 |
---|---|---|
Datameer X is now able to connect to a SFTP server that is hidden behind a proxy with the following properties: #proxy settings to connect a SFTP server hidden by the proxy das.network.proxy.sftp.type=[HTTP] das.network.proxy.sftp.host=<host ip> das.network.proxy.sftp.port=<port> das.network.proxy.sftp.username=<username> das.network.proxy.sftp.password=<password> | Bug fixed. |
5 | Workbook: Double-click to submit in credentials dialog can lead to an error | DAP-40487 |
---|---|---|
Adding a datasource to a session-based import job doesn't lead to an error any more. | Bug fixed. |
6 | Admin: Health Check report sometimes does not update correctly | DAP-40509 |
---|---|---|
When clicking the "Start" button from the Workbook Health Check the indicator now shows the correct status of the Workbook Health Check. | Bug fixed. |
7 | Datameer X Upgrade: When upgrading from Datameer X 6.4.13 to 7.5.10 several workbooks stuck at 'Registering operations for sheet' phase | DAP-40534 |
---|---|---|
When upgrading the workbooks don't stuck anymore because of patches in *.jar libraries, e.g. big workbooks with sophisticated logic. | Bug fixed. |
8 | Import Job: The partition dialog in 'Browse all Data' does not close when clicking "Select" | DAP-40536 |
---|---|---|
The dialog now closes again when clicking "Select". | Bug fixed. |
9 | FileBrowser: Browser list-files slow performance when a folder contains workbooks with data-driven scheduling | DAP-40544 |
---|---|---|
The backend query was optimized to minimize to response time of folders within the FileBrowser that contain artifacts with a long chain of upstream dependencies, e.g. workbooks that have 10 to 15 layers of parent artifacts. | Bug fixed. |
10 | Import Job: Fix intermittent failing test for email format validation | DAP-40551 |
---|---|---|
The test for the email validation now works as expected and detects not accepted email addresses, e.g. within an import job configuration. | Bug fixed. |
11 | Import: Azure Databricks import fails | DAP-40604 |
---|---|---|
An Azure Databricks import now works as expected. | Bug fixed. |
12 | Admin: Workbook Health Check is broken | DAP-40606 |
---|---|---|
The Workbook Health Check from the Admin tab now works properly again. | Bug fixed. |
13 | Export/ Hive: Data is being written as 'TEXT' format when duplicating an export job and setting a new table and data type | DAP-40655 |
---|---|---|
When duplicating a Hive export job and changing its settings, the changes are respected without any changes of data type, output format etc. in the new Hive table | Bug fixed. |
14 | Setup: Users without submitted license agreement can not operate Datameer | DAP-40681 |
---|---|---|
The license agreement flag 'force.license-agreement' is set to 'TRUE' and the internal user management is configured to allow newly created users to login and accept the terms of licence to operate Datameer. | Bug fixed. |
15 | Setup: C3p0 connection pool exhaustion | DAP-40687 |
---|---|---|
After disabling the statement caching no c3p0 warnings about pool problems appear anymore. | Bug fixed. |
16 | Performance: Housekeeping Service MySQL CPU drain | DAP-40604 |
---|---|---|
A new default property 'housekeeping.run.task-attempts-per-run' is set to a value of '50' per default to decrease the performance of the Housekeeping Service significantly when it is to low. | Bug fixed. |
17 | Workbook: Prevent Out of Memory errors when handling workbook circular dependencies | DAP-40694 |
---|---|---|
Out of Memory errors are now prevented by changing buggy code. | Bug fixed. |
18 | Workbook: Prevent Stack Overflow errors when handling workbook circular dependencies | DAP-40696 |
---|---|---|
Stack Overflow errors are now prevented by changing buggy code. | Bug fixed. |
19 | Performance: Dependencies graph performance is low | DAP-40727 |
---|---|---|
The performance for using the 'All Dependencies' and 'Depends On' function in the File Browser is now improved. | Bug fixed. |
Datameer X 10.1.4.1
Improvements
1 | Datameer X White Labeling: White labeling Datameer X for OEM | DAP-40550 |
---|---|---|
Several default properties were created to change the Datameer X UI and URL to have the Datameer X application white labeled. Furthermore the Datameer X distribution allows to change the small and big logo as well as favicons. |
2 | Datameer X White Labeling: Help links shouldn't be shown if a property pointing to the public documentation is invalid or empty | DAP-40386 |
---|---|---|
Links to the Datameer X public documentation are not shown within the white labeled Datameer X UI by default. The links are only shown when the property 'doc url' is set. This is also valid for the 'Help' menu. |
3 | FileBrowser: Optimize BrowserService calls for better performance | DAP-40565 |
---|---|---|
The BrowserService is now optimized not to access "WB#getSheets" and "WB#isSync". |
Bug Fixes
1 | Upgrading Datameer X After an upgrade several workbooks stuck at the "Registering operations for sheet" phase. | DAP-40534 |
---|---|---|
Datameer X workbooks don't get stuck anymore. | Bug fixed. |
2 | Performance: Browser list-files show a slow performance when the folder contains workbooks with data-driven scheduling | DAP-40544 |
---|---|---|
Browser list-files work with regular performance when a folder contains workbooks with data-driven scheduling. | Bug fixed. |
Datameer X 10.1.4
Improvements
1 | Supported Hadoop Distributions: Add Support for Cloudera CDP-DC/ Cloudera Runtime 7.1.1 | DAP-40110 |
---|---|---|
Datameer X now supports Cloudera Runtime version 7.1.1. |
2 | Workbook: Change 'SHIFTTIMEZONE' function arguments' name | DAP-40199 |
---|---|---|
The description of the function arguments has been changed in the formula builder to prevent ambiguities. There is no behavior change of the functionality of the function itself. |
3 | Setup: Enable the 'DAS_USER' property in the 'etc/das-env.sh' file to reflect the assigned service account | DAP-40247 |
---|---|---|
Adjusting this parameter is critical in saving the Admin from themselves by preventing an accidental startup of Datameer X without first assuming the proper service account. The property 'DAS_USER' must be enabled in the 'etc/das-env.sh' file. |
4 | Workbook: Extend variable value length | DAP-40265 |
---|---|---|
Within a fresh Datameer X distribution, it is now possible to create variable values with more than 255 characters. |
5 | Performance: Validate Datameer X license status once a day only | DAP-40272 |
---|---|---|
A Datameer X license now gets checked with the first user authentication or scheduled job run after midnight (local time) and is valid 24 h afterwards. |
6 | Workbook: Validate cross joins for SQL sheets | DAP-40276 |
---|---|---|
Datameer X validates queries of SQL sheets and notifies an analyst if he accidentally created a cross join in a 'FROM' section. Furthermore Datameer X allows the explicit usage of a 'cross join' syntax. The validation can be enabled by setting the 'sql-cross-join.enabled property' in the 'default.properties' configuration file. |
7 | Setup: Document MariaDB as a supported database dialect for Datameer's metadata database | DAP-40293 |
---|---|---|
MariaDB is now listed as a valid setting for the 'system.property.db.mode'. |
8 | Supported Hadoop Distributions: Add Support for Amazon EMR v5.30.0 | DAP-40355 |
---|---|---|
Datameer X now supports Amazon EMR v5.30.0. |
9 | Supported Hadoop Distributions: Add Support for Amazon EMR v6.0.0 | DAP-40354 |
---|---|---|
Datameer X now supports Amazon EMR v6.0.0. |
Bug Fixes
1 | Import/ Export: Add tooltips for long column names in Full Data and Export Wizard step 'Choose Sheet' | DAP-38627 |
---|---|---|
Datameer X now renders tooltips for long column names. | Bug fixed. |
2 | Import Job: Browse all data, and job details are opened in separate tabs | DAP-39613 |
---|---|---|
Datameer X stays in the same browser tab during navigation after running an import job. | Bug fixed. |
3 | Import Job: 'DelegateImportFormat' and 'SequenceFileType' create an empty configuration instead of using the provided one | DAP-39884 |
---|---|---|
Datameer X now validates for properly configured 'io.serializations' settings for custom sequence file formats. | Bug fixed. |
4 | Import/ Export: Error during an ImportJob creation | DAP-40148 |
---|---|---|
While creating a BigQuery-based ImportJob in a Google BigQuery connection, no 'IllegalArgumentException' error is shown in the 'Data Details' step anymore. | Bug fixed. |
5 | Workbook: Can not rename a source column from a SQL joined sheet | DAP-40158 |
---|---|---|
Renaming a source column of a SQL sheet doesn't lead to an error any more. | Bug fixed. |
6 | Import: Can't import an AVRO file via HDFS HA Connection | DAP-40172 |
---|---|---|
Importing AVRO files from HA HDFS connections now works. | Bug fixed. |
7 | Setup: Datameer X doesn't clean up temporary data | DAP-40242 |
---|---|---|
Datameer X does a clean-up of its temporary folder during a start-up as long it is configured under Datameer's installation directory, but keeps it untouched otherwise. | Bug fixed. |
8 | HiveServer2 Plug-in: Enable support of mid-column schema changes of an underlying Hive table (AVRO & Parquet) | DAP-40267 |
---|---|---|
Datameer's Hive plug-in is supporting mid-column schema changes for Parquet and AVRO storage formats now. | Bug fixed. |
9 | File Browser: Details of an error dialog box opens a new browser tab | DAP-40292 |
---|---|---|
Error details are now rendered in the current browser tab. | Bug fixed. |
10 | Import Job: An import job is still referenced in a workbook after deleted the datasource in the workbook | DAP-40321 |
---|---|---|
When deleting a datasource, the current workbook holds no further references to it anymore. | Bug fixed. |
11 | Workbook: Workbook job run fails with an SQL sheet | DAP-40327 |
---|---|---|
The 'deploy.mode' is passed in for SQL sheets correctly now and the job run doesn't fail any longer. | Bug fixed. |
12 | Administration: The "User can see every file and folder" permission grants 'View Full Results' without an explicit mention. | DAP-40366 |
---|---|---|
Using this permission does not grant access to view data, only to see the files and folders. The ability to grant view full results became be a separate option to modify underneath. | Bug fixed. |
13 | Connection: Issue connecting to Amazon Athena | DAP-40372 |
---|---|---|
The connection to Athena now works when in the custom properties the 'das.jdbc.import.transaction-isolation=TRANSACTION_NONE' property is set. | Bug fixed. |
14 | File Browser: Abort in 'Duplicate Folder' or 'Paste Folder' does not work | DAP-40374 |
---|---|---|
Clicking the 'Abort' button while duplicating or pasting a folder now cancels the operation. | Bug fixed. |
15 | REST API: REST API v1 vs v2 - inconsistent handling of workbook UUIDs | DAP-40400 |
---|---|---|
The REST API v2 will generate a new random UUID during POST (while a valid UUID is required in the payload, e.g. '00000000-0000-0000-0000-000000000000' can be used as a template). | Bug fixed. |
16 | Plug-ins: Errors are swallowed, making it look like the plug-in was successfully initialized | DAP-40406 |
---|---|---|
A plug-in will not get registered if there has been an error during plug-in initialization. | Bug fixed. |
17 | HiveServer2: A workbook that contains a datalink drops the records after a schema change of the underlying Parquet table | DAP-40448 |
---|---|---|
Datameer X now imports all records from before and after a schema change. | Bug fixed. |
18 | Job Status Notification: Validation of the e-mail address according to RFC2822 (in detail: e-mail can contain "&") | DAP-40453 |
---|---|---|
Job status notifications for import, export and workbooks jobs can be send via e-mail addresses that contain chars like '&' from now on. | Bug fixed. |
19 | Support Engineer Report: The property 'SupportEngineerReportController.filterFilesLastNDays()' must cut at midnight from the system default timezone, not UTC | DAP-40467 |
---|---|---|
The 'Support Engineer Report' now respects the system default timezone. | Bug fixed. |
20 | Plug-in Tez: 'CONSUMED_BYTES/RECORDS' is incorrect in some cases | DAP-40471 |
---|---|---|
The plug-in now reports the counters for all (aliased) inputs in the dataset. | Bug fixed. |
21 | Plug-in Tez: A 'job-conf.xml' debug artifact should be written only once per Tez job, not for each vertex | DAP-40475 |
---|---|---|
The 'job-conf.xml' debug artifact is only written once for a job now, therefore rate limits like in GCP/ GCS don't occur. | Bug fixed. |
Datameer X 10.1.3.1
New Features
1 | Supported Hadoop Distributions: Add Support for Cloudera CDP / Cloudera Runtime 7.0(.3) | DAP-39597 |
---|---|---|
Cloudera Runtime 7.0(.3) is now supported. |
Improvements
1 | Connection: Step 'Save' in connection edit mode - show a list of impacted artifacts when the authentication mode is set to 'SESSION_BASED' | DAP-39087 |
---|---|---|
A list of the impacted artifacts is now displayed when the authentication mode is set to 'SESSION_BASED' in the section 'Save' during connections. |
2 | Storybook: Enable new authentication dialog to request credentials | DAP-39088 |
---|---|---|
In the storybook a new React component is enabled which requests either user 'Name', 'Password' or 'SSH Key' depending on the used connection. |
3 | Supported Hadoop Distributions: Add support for Hortonworks HDP 3.1.5 | DAP-39819 |
---|---|---|
Datameer X now supports Hortonworks HDP 3.1.5. |
4 | Tableau Export Connection: Parallel TDE/ Hyper file generation for Tableau Server exports | DAP-39846 |
---|---|---|
Setting the property 'das.splitting.disable-combining=true' while configuring a Tableau connection runs the jobs significantly faster. It is verified that an export of a sheet to the Tableau server launches multiple tasks. |
5 | Supported Hadoop Distributions: Add support for Cloudera CDH 6.3.3 | DAP-39933 |
---|---|---|
Datameer X now supports Cloudera CDH 6.3.3. |
6 | Import job/ DataLink Wizard: Request credentials while edit/ create | DAP-39935 |
---|---|---|
The property 'enable.authenticationOnJobRun=true' activates the session base authentication feature. If a connection is configured as 'SESSION_BASED' and the authentication is already entered, the wizard will not show the ask for the credentials group. If a connection switches from 'PROVIDED/TEMPORARY' credentials to 'SESSION BASED' and edit/ save the import job, a check after saving in the database table 'job_configuration_property' the properties 'datastore.user_name' and 'datastore.password' must be deleted/ not exist for a session based connection import/datalink job. Furthermore a check after saving in the database table 'dap_job_configuration' column 'schedule_enabled' must be true (1) for 'PROVIDED' and 'TEMPORARY' and false (0) for 'SESSION BASED'. |
7 | Workbook: Display a "Save" button in toolbar | DAP-40047 |
---|---|---|
You can now click on a 'Save' button in the UI toolbar again to safe changes within a workbook. |
8 | Workbook: Workbook Saving - Default to working path | DAP-40055 |
---|---|---|
When closing a workbook for the first time, the workbook is saved in the folder it was created from and not in home folder upon exit. A workbook message does not read "saved under home folder" but using the actual path where the workbook is stored. When closing the File Browser the workbook is saved in the folder it was created from. |
9 | Supported Hadoop Distributions: Add HBase 2 support for Cloudera CDH 6 versions | DAP-40109 |
---|---|---|
Datameer X now supports HBase 2 support for Cloudera CDH 6 versions. |
10 | Tableau Plug-in: Tableau Plug-in allow TDE/ Hyper exports to be setup from non-Linux systems | DAP-40147 |
---|---|---|
The plug-in for Tableau now allows TDE/ Hyper exports to be setup from non-Linux systems. |
11 | Workbook Settings: Disable scheduling if "request credentials on job trigger" condition is fulfilled | DAP-39939 |
---|---|---|
The scheduling menu is disabled if "request credentials on job trigger" conditions are fulfilled. A warning dialog is displayed why the 'Trigger' section is disabled. The REST v2 artifacts for job configurations now contain a new state if scheduling is enabled/ disabled. |
12 | REST API: Enable back authentication for REST API call via session cookies | DAP-40012 |
---|---|---|
Datameer X has two new security models for for REST API calls via session cookies which can be switched with the property 'enable.rest.csrf.checking=true/false'. The standard mode requires a CSRF token while the loose mode allows the authentication without a CSRF token. |
Bug Fixes
1 | Column Inspector: 'Undefined' shown as a formula string in case of empty columns | DAP-39529 |
---|---|---|
Empty columns do not show formulas in the column inspector any more. | Bug fixed. |
2 | Tableau Export: Tableau ExportJobs performance issue | DAP-40034 |
---|---|---|
The property 'das.splitting.disable-combining=true' while configuring a Tableau connection runs the jobs significantly faster and multiple tasks can be launched. | Bug fixed. |
3 | Workbook: Duplicating or copy/ pasting a workbook returns an error | DAP-40067 |
---|---|---|
Workbooks can be copied and pasted without an error again. | Bug fixed. |
4 | Workbook: 'EXPAND' function column position shows incorrect result for 'GROUPROWNUMBER' | DAP-40087 |
---|---|---|
Using the 'EXPAND(LIST)' function shows the correct sequence for 'GROUPROWNUMBER'. | Bug fixed. |
5 | Display key columns first flag causes 'IllegalStateException' in a 'Join Sheet' | DAP-40121 |
---|---|---|
Displaying the key columns first flag doesn't cause exceptions in a 'Join Sheet' any more. | Bug fixed. |
6 | Workbook: Can't rename a column in the 'Join Sheet' because of a 'RegularRequirementNotMetFailure' | DAP-40130 |
---|---|---|
Column renaming in a 'Joint Sheet' is possible because the broken references check implemented for transformed sheets doesn't affect renaming at other sheet types. | Bug fixed. |
7 | Upgrading Datameer X 7.5.x to 10.1.2 workbook upgrade conversion breakage | DAP-40140 |
---|---|---|
Column changes are propagated to all depended workbooks and sheet, not only when upgrading. | Bug fixed. |
8 | Avro Import: 'NoSuchMethodError' with Cloudera CDH-5.14.4 (Hadoop distribution jackson jar binary incompatible) | DAP-40142 |
---|---|---|
Importing Avro files works on Cloudera CDH-5.14.4 distributions. | Bug fixed. |
9 | Workbook: Datameer X OOM crash when viewing sorted sheet data | DAP-40151 |
---|---|---|
The new property 'das.data-download.merge-sort.max-total-file-size' is introduced to control (volume based) if the sorting will happen in full data view or not. The default value is 256MB. | Bug fixed. |
10 | Datameer X Mode: Prevent mixed content issues if Datameer X is configured in SSL mode | DAP-40176 |
---|---|---|
The button "New System" within 'External Systems' is available for entitled users such as an Admin in a SSL enabled installation, too. | Bug fixed. |
11 | Upgrading Process: After upgrade the search index could be wrong or missing | DAP-39528 |
---|---|---|
After the upgrading process, the old search index is deleted and a new index created. | Bug fixed. |
12 | Workbook: Pivot Sheet - While creating/ editing a Pivot sheet the buttons 'Toggle Column' and 'Go To Line' are not disabled | DAP-39770 |
---|---|---|
When a Pivot sheet is created or an existing Pivot sheet is edited in a workbook, the button 'Toggle Column' is disabled. When a Pivot sheet is created or edited in a workbook, the button 'Go To Line' is disabled. On Pivot sheet while being in Pivot preview mode clicking on 'Split', 'De-Duplicate' or 'Encode Column', the toolbar buttons dismiss the Pivot review mode. | Bug fixed. |
13 | Workbook: 'Show Dependencies' is very slow and keeps loading | DAP-40023 |
---|---|---|
The 'Dependencies' are now displayed in an appropriate amount of time when clicking the icon. | Bug fixed. |
14 | Workbook: It is not possible to use a column in a join if the column was in an error state | DAP-40052 |
---|---|---|
You can use a column in a join if the column was in an error state before and switch sheet was involved. | Bug fixed. |
15 | Datameer X Grid Mode: EMR - Clustername nor ID are validated while saving the grid mode. | DAP-40100 |
---|---|---|
While saving an EMR connection configuration, the clustername and ID are now validated. | Bug fixed. |
16 | Neebo to Datameer X Accessibility: Data Warehouse plug-in shouldn't restrict the access to file descriptors by 'HADOOP_CLUSTER_ACCESS' capability | DAP-40201 |
---|---|---|
Datameer X data is accessible via Neebo since a user needs 'IMPORTJOB_DOWNLOAD' and 'WORKBOOK_DOWNLOAD' capabilities are guaranteed with at least the role 'ANALYST'. | Bug fixed. |
17 | Export Job: Options for 'On a schedule' are missing | DAP-40227 |
---|---|---|
When creating an export job, for the schedule method "on a schedule" several options are available, e.g. hourly, monthly. | Bug fixed. |
Datameer X 10.1.2
New Features
1 | Add REST API call to look up what a UUID's properties are. | DAP-31229 |
---|---|---|
For a UUID Datameer X provides the JSON containing the information about the artifact analog to a REST READ call for an artifact ID. |
Improvements
1 | Connection: In wizard section 'details' validate and store authentication on a session if the authentication session-based option is selected | DAP-39334 |
---|---|---|
Authentication data is now validated and stored per session in the 'Connection Details' section within the connection wizard. |
2 | Connection: Show selected authentication mode on 'Connection Details' | DAP-39335 |
---|---|---|
Datameer X now shows the authentication mode on the 'Connection Details' page if the connection type provides such options (JDBC/MySQL, SSH, SFTP connections). |
3 | Connection: Implement a controller for session-based connection authentication | DAP-39936 |
---|---|---|
A new service checks the session id and connection id for session-based authentication connections. |
Bug Fixes
1 | Workbook: Inconsistency in error messages for workbooks without a shared source | DAP-38108 |
---|---|---|
Error messages are adjusted and are now consistent for errors when working with a shared workbook without a shared source. | Bug fixed. |
2 | Mysql 8.x execute mysql-init.sql fails - no Grant options allowed | DAP-39867 |
---|---|---|
Datameer X has two new MySQL scripts for running MySQL 5.x and MySQL 8.x databases. | Bug fixed. |
3 | Security: After a failed login attempt, Datameer X exposes the LDAP query to fetch users | DAP-39953 |
---|---|---|
The error message is shown again as expected and does not expose the LDAP query to fetch users anymore. | Bug fixed. |
4 | Google Cloud Storage is bundled with the wrong Hadoop library | DAP-39979 |
---|---|---|
The Google Cloud Storage connector is now bundled with all Hadoop 3 versions. | Bug fixed. |
5 | Infographics: DataSetsController REST API copies full workbook or import job data to DFS (EhCache), therefore the conductor disk space is filling up | DAP-39990 |
---|---|---|
The DFSCache button on the developer's page is working again and data is now cleaned up as expected. | Bug fixed. |
6 | DevController: Cache Monitor 'Clear Cache' doesn't purge data | DAP-40005 |
---|---|---|
The DFS cache is now cleared when clicking the "Clear Cache" button. | Bug fixed. |
7 | Workbook: Filtering on a column with the 'DENULLIFY' function causes an 'IllegalArgumentException' | DAP-40017 |
---|---|---|
The Datameer X function 'DENULLIFY' works as expected again. | Bug fixed. |
8 | FileBrowser: Increase polling interval for thr file browser's dependency view | DAP-40038 |
---|---|---|
Due to performance, the polling interval while working with the file browser's dependencies view is now increased to 100 sec. | Bug fixed. |
9 | Workbook: Typo in backup and restore error message | DAP-40046 |
---|---|---|
The error messages for duplicating and copying/pasting a shared workbook without sharing the data source is now correct. | Bug fixed. |