INFO
Within these release notes Datameer provides information and resolutions on improvements, bug fixes, and visual changes for 10.1.x maintenance release versions.
Visit Datameer's Help Center for additional information or contact your customer service representative.
Datameer 10.1.4.1
Improvements
1 | White labeling Datameer for OEM | DAP-40550 |
|---|---|---|
2 | Help links shouldn't be shown if property pointing to docu is invalid or empty | DAP-40386 |
|---|---|---|
3 | FileBrowser: Optimize BrowserService calls for better performance | DAP-40565 |
|---|---|---|
The BrowserService is now optimized not to access "WB#getSheets" and "WB#isSync". |
Bug fixes
1 | Upgrading Datameer: After an upgrade several workbooks stuck at the "Registering operations for sheet" phase. | DAP-40534 |
|---|---|---|
Datameer workbooks don't get stuck anymore. |
|
2 | Performance: Browser list-files show a slow performance when the folder contains workbooks with data-driven scheduling | DAP-40544 |
|---|---|---|
Browser list-files work with regular performance when a folder contains workbooks with data-driven scheduling. |
|
Datameer 10.1.4
Improvements
1 | Supported Hadoop Distributions: Add Support for Cloudera CDP-DC/ Cloudera Runtime 7.1.1 | DAP-40110 |
|---|---|---|
Datameer now supports Cloudera Runtime version 7.1.1. |
2 | Workbook: Change 'SHIFTTIMEZONE' function arguments' name | DAP-40199 |
|---|---|---|
The description of the function arguments has been changed in the formula builder to prevent ambiguities. There is now behavior change of the functionality of the function itself. |
3 | Setup: Enable the 'DAS_USER' property in the 'etc/das-env.sh' file to reflect the assigned service account | DAP-40247 |
|---|---|---|
Adjusting this parameter is critical in saving the Admin from themselves by preventing an accidental startup of Datameer without first assuming the proper service account. The property 'DAS_USER' must be enabled in the 'etc/das-env.sh' file. |
| 4 | Workbook: Extend variable value length | DAP-40265 |
|---|---|---|
| Within a fresh Datameer distribution, it is now possible to create variable values with more than 255 characters. |
| 5 | Performance: Validate Datameer license status once a day only | DAP-40272 |
|---|---|---|
| A Datameer license now gets checked with the first user authentication or scheduled job run after midnight (local time) and is valid 24 h afterwards. |
| 6 | Workbook: Validate cross joins for SQL sheets | DAP-40276 |
|---|---|---|
Datameer validates queries of SQL sheets and notifies an analyst if he accidentally created a cross join in a 'FROM' section. Furthermore Datameer allows the explicit usage of a 'cross join' syntax. The validation can be enabled by setting the 'sql-cross-join.enabled property' in the 'default.properties' configuration file. |
| 7 | Setup: Document MariaDB as a supported database dialect for Datameer's metadata database | DAP-40293 |
|---|---|---|
| MariaDB is now listed as a valid setting for the 'system.property.db.mode'. |
| 8 | Supported Hadoop Distributions: Add Support for Amazon EMR v5.30.0 | DAP-40355 |
|---|---|---|
| Datameer now supports Amazon EMR v5.30.0. |
| 9 | Supported Hadoop Distributions: Add Support for Amazon EMR v6.0.0 | DAP-40354 |
|---|---|---|
| Datameer now supports Amazon EMR v6.0.0. |
Bug fixes
1 | Import/ Export: Add tooltips for long column names in Full Data and Export Wizard step 'Choose Sheet' | DAP-38627 |
|---|---|---|
Datameer now renders tooltips for long column names. |
|
2 | Import Job: Browse all data, and job details are opened in separate tabs | DAP-39613 |
|---|---|---|
Datameer stays in the same browser tab during navigation after running an import job. |
|
3 | Import Job: 'DelegateImportFormat' and 'SequenceFileType' create an empty configuration instead of using the provided one | DAP-39884 |
|---|---|---|
Datameer now validates for properly configured 'io.serializations' settings for custom sequence file formats. |
|
4 | Import/ Export: Error during an ImportJob creation | DAP-40148 |
|---|---|---|
While creating a BigQuery-based ImportJob in a Google BigQuery connection, no 'IllegalArgumentException' error is shown in the 'Data Details' step anymore. |
|
5 | Workbook: Can not rename a source column from a SQL joined sheet | DAP-40158 |
|---|---|---|
Renaming a source column of a SQL sheet doesn't lead to an error any more. |
|
6 | Import: Can't import an AVRO file via HDFS HA Connection | DAP-40172 |
|---|---|---|
Importing AVRO files from HA HDFS connections now works. |
|
7 | Setup: Datameer doesn't clean up temporary data | DAP-40242 |
|---|---|---|
Datameer does a clean-up of its temporary folder during a start-up as long it is configured under Datameer's installation directory, but keeps it untouched otherwise. |
|
8 | HiveServer2 Plug-in: Enable support of mid-column schema changes of an underlying Hive table (AVRO & Parquet) | DAP-40267 |
|---|---|---|
Datameer's Hive plug-in is supporting mid-column schema changes for Parquet and AVRO storage formats now. |
|
9 | File Browser: Details of an error dialog box opens a new browser tab | DAP-40292 |
|---|---|---|
Error details are now rendered in the current browser tab. |
|
10 | Import Job: An import job is still referenced in a workbook after deleted the datasource in the workbook | DAP-40321 |
|---|---|---|
When deleting a datasource, the current workbook holds no further references to it anymore. |
|
11 | Workbook: Workbook job run fails with an SQL sheet | DAP-40327 |
|---|---|---|
The 'deploy.mode' is passed in for SQL sheets correctly now and the job run doesn't fail any longer. |
|
12 | Administration: The "User can see every file and folder" permission grants 'View Full Results' without an explicit mention. | DAP-40366 |
|---|---|---|
Using this permission does not grant access to view data, only to see the files and folders. The ability to grant view full results became be a separate option to modify underneath. |
|
13 | Connection: Issue connecting to Amazon Athena | DAP-40372 |
|---|---|---|
The connection to Athena now works when in the custom properties the 'das.jdbc.import.transaction-isolation=TRANSACTION_NONE' property is set. |
|
14 | File Browser: Abort in 'Duplicate Folder' or 'Paste Folder' does not work | DAP-40374 |
|---|---|---|
Clicking the 'Abort' button while duplicating or pasting a folder now cancels the operation. |
|
15 | REST API: REST API v1 vs v2 - inconsistent handling of workbook UUIDs | DAP-40400 |
|---|---|---|
The REST API v2 will generate a new random UUID during POST (while a valid UUID is required in the payload, e.g. '00000000-0000-0000-0000-000000000000' can be used as a template). |
|
16 | Plug-ins: Errors are swallowed, making it look like the plug-in was successfully initialized | DAP-40406 |
|---|---|---|
A plug-in will not get registered if there has been an error during plug-in initialization. |
|
17 | HiveServer2: A workbook that contains a datalink drops the records after a schema change of the underlying Parquet table | DAP-40448 |
|---|---|---|
Datameer now imports all records from before and after a schema change. |
|
18 | Job Status Notification: Validation of the e-mail address according to RFC2822 (in detail: e-mail can contain "&") | DAP-40453 |
|---|---|---|
Job status notifications for import, export and workbooks jobs can be send via e-mail addresses that contain chars like '&' from now on. |
|
19 | Support Engineer Report: The property 'SupportEngineerReportController.filterFilesLastNDays()' must cut at midnight from the system default timezone, not UTC | DAP-40467 |
|---|---|---|
The 'Support Engineer Report' now respects the system default timezone. |
|
20 | Plug-in Tez: 'CONSUMED_BYTES/RECORDS' is incorrect in some cases | DAP-40471 |
|---|---|---|
The plug-in now reports the counters for all (aliased) inputs in the dataset. |
|
21 | Plug-in Tez: A 'job-conf.xml' debug artifact should be written only once per Tez job, not for each vertex | DAP-40475 |
|---|---|---|
The 'job-conf.xml' debug artifact is only written once for a job now, therefore rate limits like in GCP/ GCS don't occur. |
|
Datameer 10.1.3.1
New Features
1 | Supported Hadoop Distributions: Add Support for Cloudera CDP / Cloudera Runtime 7.0(.3) | DAP-39597 |
|---|---|---|
Cloudera Runtime 7.0(.3) is now supported. |
Improvements
1 | Connection: Step 'Save' in connection edit mode - show a list of impacted artifacts when the authentication mode is set to 'SESSION_BASED' | DAP-39087 |
|---|---|---|
A list of the impacted artifacts is now displayed when the authentication mode is set to 'SESSION_BASED' in the section 'Save' during connections. |
2 | Storybook: Enable new authentication dialog to request credentials | DAP-39088 |
|---|---|---|
In the storybook a new React component is enabled which requests either user 'Name', 'Password' or 'SSH Key' depending on the used connection. |
3 | Supported Hadoop Distributions: Add support for Hortonworks HDP 3.1.5 | DAP-39819 |
|---|---|---|
Datameer now supports Hortonworks HDP 3.1.5. |
| 4 | Tableau Export Connection: Parallel TDE/ Hyper file generation for Tableau Server exports | DAP-39846 |
|---|---|---|
| Setting the property 'das.splitting.disable-combining=true' while configuring a Tableau connection runs the jobs significantly faster. It is verified that an export of a sheet to the Tableau server launches multiple tasks. |
| 5 | Supported Hadoop Distributions: Add support for Cloudera CDH 6.3.3 | DAP-39933 |
|---|---|---|
| Datameer now supports Cloudera CDH 6.3.3. |
| 6 | Import job/ DataLink Wizard: Request credentials while edit/ create | DAP-39935 |
|---|---|---|
The property 'enable.authenticationOnJobRun=true' activates the session base authentication feature. If a connection is configured as 'SESSION_BASED' and the authentication is already entered, the wizard will not show the ask for the credentials group. If a connection switches from 'PROVIDED/TEMPORARY' credentials to 'SESSION BASED' and edit/ save the import job, a check after saving in the database table 'job_configuration_property' the properties 'datastore.user_name' and 'datastore.password' must be deleted/ not exist for a session based connection import/datalink job. Furthermore a check after saving in the database table 'dap_job_configuration' column 'schedule_enabled' must be true (1) for 'PROVIDED' and 'TEMPORARY' and false (0) for 'SESSION BASED'. |
| 7 | Workbook: Display a "Save" button in toolbar | DAP-40047 |
|---|---|---|
| You can now click on a 'Save' button in the UI toolbar again to safe changes within a workbook. |
| 8 | Workbook: Workbook Saving - Default to working path | DAP-40055 |
|---|---|---|
| When closing a workbook for the first time, the workbook is saved in the folder it was created from and not in home folder upon exit. A workbook message does not read "saved under home folder" but using the actual path where the workbook is stored. When closing the File Browser the workbook is saved in the folder it was created from. |
| 9 | Supported Hadoop Distributions: Add HBase 2 support for Cloudera CDH 6 versions | DAP-40109 |
|---|---|---|
| Datameer now supports HBase 2 support for Cloudera CDH 6 versions. |
| 10 | Tableau Plug-in: Tableau Plug-in allow TDE/ Hyper exports to be setup from non-Linux systems | DAP-40147 |
|---|---|---|
| The plug-in for Tableau now allows TDE/ Hyper exports to be setup from non-Linux systems. |
| 11 | Workbook Settings: Disable scheduling if "request credentials on job trigger" condition is fulfilled | DAP-39939 |
|---|---|---|
| The scheduling menu is disabled if "request credentials on job trigger" conditions are fulfilled. A warning dialog is displayed why the 'Trigger' section is disabled. The REST v2 artifacts for job configurations now contain a new state if scheduling is enabled/ disabled. |
| 12 | REST API: Enable back authentication for REST API call via session cookies | DAP-40012 |
|---|---|---|
| Datameer has two new security models for for REST API calls via session cookies which can be switched with the property 'enable.rest.csrf.checking=true/false'. The standard mode requires a CSRF token while the loose mode allows the authentication without a CSRF token. |
Bug fixes
1 | Column Inspector: 'Undefined' shown as a formula string in case of empty columns | DAP-39529 |
|---|---|---|
Empty columns do not show formulas in the column inspector any more. |
|
2 | Tableau Export: Tableau ExportJobs performance issue | DAP-40034 |
|---|---|---|
The property 'das.splitting.disable-combining=true' while configuring a Tableau connection runs the jobs significantly faster and multiple tasks can be launched. |
|
3 | Workbook: Duplicating or copy/ pasting a workbook returns an error | DAP-40067 |
|---|---|---|
Workbooks can be copied and pasted without an error again. |
|
| 4 | Workbook: 'EXPAND' function column position shows incorrect result for 'GROUPROWNUMBER' | DAP-40087 |
|---|---|---|
| Using the 'EXPAND(LIST)' function shows the correct sequence for 'GROUPROWNUMBER'. |  Bug fixed. |
| 5 | Display key columns first flag causes 'IllegalStateException' in a 'Join Sheet' | DAP-40121 |
|---|---|---|
Displaying the key columns first flag doesn't cause exceptions in a 'Join Sheet' any more. | Bug fixed. |
| 6 | Workbook: Can't rename a column in the 'Join Sheet' because of a 'RegularRequirementNotMetFailure' | DAP-40130 |
|---|---|---|
| Column renaming in a 'Joint Sheet' is possible because the broken references check implemented for transformed sheets doesn't affect renaming at other sheet types. | Bug fixed. |
| 7 | Upgrading Datameer: 7.5.x to 10.1.2 workbook upgrade conversion breakage | DAP-40140 |
|---|---|---|
| Column changes are propagated to all depended workbooks and sheet, not only when upgrading. | Bug fixed. |
| 8 | Avro Import: 'NoSuchMethodError' with Cloudera CDH-5.14.4 (Hadoop distribution jackson jar binary incompatible) | DAP-40142 |
|---|---|---|
| Importing Avro files works on Cloudera CDH-5.14.4 distributions. | Bug fixed. |
| 9 | Workbook: Datameer OOM crash when viewing sorted sheet data | DAP-40151 |
|---|---|---|
| The new property 'das.data-download.merge-sort.max-total-file-size' is introduced to control (volume based) if the sorting will happen in full data view or not. The default value is 256MB. | Bug fixed. |
| 10 | Datameer Mode: Prevent mixed content issues if Datameer is configured in SSL mode | DAP-40176 |
|---|---|---|
| The button "New System" within 'External Systems' is available for entitled users such as an Admin in a SSL enabled installation, too. | Bug fixed. |
| 11 | Upgrading Process: After upgrade the search index could be wrong or missing | DAP-39528 |
|---|---|---|
| After the upgrading process, the old search index is deleted and a new index created. | Bug fixed. |
| 12 | Workbook: Pivot Sheet - While creating/ editing a Pivot sheet the buttons 'Toggle Column' and 'Go To Line' are not disabled | DAP-39770 |
|---|---|---|
When a Pivot sheet is created or an existing Pivot sheet is edited in a workbook, the button 'Toggle Column' is disabled. When a Pivot sheet is created or edited in a workbook, the button 'Go To Line' is disabled. On Pivot sheet while being in Pivot preview mode clicking on 'Split', 'De-Duplicate' or 'Encode Column', the toolbar buttons dismiss the Pivot review mode. | Bug fixed. |
| 13 | Workbook: 'Show Dependencies' is very slow and keeps loading | DAP-40023 |
|---|---|---|
| The 'Dependencies' are now displayed in an appropriate amount of time when clicking the icon. | Bug fixed. |
| 14 | Workbook: It is not possible to use a column in a join if the column was in an error state | DAP-40052 |
|---|---|---|
| You can use a column in a join if the column was in an error state before and switch sheet was involved. | Bug fixed. |
| 15 | Datameer Grid Mode: EMR - Clustername nor ID are validated while saving the grid mode. | DAP-40100 |
|---|---|---|
| While saving an EMR connection configuration, the clustername and ID are now validated. | Bug fixed. |
| 16 | Neebo to Datameer Accessibility: Data Warehouse plug-in shouldn't restrict the access to file descriptors by 'HADOOP_CLUSTER_ACCESS' capability | DAP-40201 |
|---|---|---|
| Datameer data is accessible via Neebo since a user needs 'IMPORTJOB_DOWNLOAD' and 'WORKBOOK_DOWNLOAD' capabilities are guaranteed with at least the role 'ANALYST'. | Bug fixed. |
| 17 | Export Job: Options for 'On a schedule' are missing | DAP-40227 |
|---|---|---|
| When creating an export job, for the schedule method "on a schedule" several options are available, e.g. hourly, monthly. | Bug fixed. |
Datameer 10.1.2
New Features
1 | Add REST API call to look up what a UUID's properties are. | DAP-31229 |
|---|---|---|
For a UUID Datameer provides the JSON containing the information about the artifact analog to a REST READ call for an artifact ID. |
Improvements
1 | Connection: In wizard section 'details' validate and store authentication on a session if the authentication session-based option is selected | DAP-39334 |
|---|---|---|
Authentication data is now validated and stored per session in the 'Connection Details' section within the connection wizard. |
2 | Connection: Show selected authentication mode on 'Connection Details' | DAP-39335 |
|---|---|---|
Datameer now shows the authentication mode on the 'Connection Details' page if the connection type provides such options (JDBC/MySQL, SSH, SFTP connections). |
3 | Connection: Implement a controller for session-based connection authentication | DAP-39936 |
|---|---|---|
A new service checks the session id and connection id for session-based authentication connections. |
Bug fixes
1 | Workbook: Inconsistency in error messages for workbooks without a shared source | DAP-38108 |
|---|---|---|
Error messages are adjusted and are now consistent for errors when working with a shared workbook without a shared source. |
|
2 | Mysql 8.x execute mysql-init.sql fails - no Grant options allowed | DAP-39867 |
|---|---|---|
Datameer has two new MySQL scripts for running MySQL 5.x and MySQL 8.x databases. |
|
3 | Security: After a failed login attempt, Datameer exposes the LDAP query to fetch users | DAP-39953 |
|---|---|---|
The error message is shown again as expected and does not expose the LDAP query to fetch users anymore. |
|
4 | Google Cloud Storage is bundled with the wrong Hadoop library | DAP-39979 |
|---|---|---|
The Google Cloud Storage connector is now bundled with all Hadoop 3 versions. |
|
| 5 | Infographics: DataSetsController REST API copies full workbook or import job data to DFS (EhCache), therefore the conductor disk space is filling up | DAP-39990 |
|---|---|---|
| The DFSCache button on the developer's page is working again and data is now cleaned up as expected. | Bug fixed. |
| 6 | DevController: Cache Monitor 'Clear Cache' doesn't purge data | DAP-40005 |
|---|---|---|
| The DFS cache is now cleared when clicking the "Clear Cache" button. | Bug fixed. |
| 7 | Workbook: Filtering on a column with the 'DENULLIFY' function causes an 'IllegalArgumentException' | DAP-40017 |
|---|---|---|
| The Datameer function 'DENULLIFY' works as expected again. | Bug fixed. |
| 8 | FileBrowser: Increase polling interval for thr file browser's dependency view | DAP-40038 |
|---|---|---|
| Due to performance, the polling interval while working with the file browser's dependencies view is now increased to 100 sec. | Bug fixed. |
| 9 | Workbook: Typo in backup and restore error message | DAP-40046 |
|---|---|---|
| The error messages for duplicating and copying/pasting a shared workbook without sharing the data source is now correct. | Bug fixed. |