Versions Compared

Version Old Version 11 New Version Current
Changes made by

Andreas Küster

Juliane Wetzel

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Info
titleINFO

Within these release notes Datameer provides information and resolutions on improvements, bug fixes, and visual changes for 10.0.x maintenance release versions.  

Visit Datameer's Help Center for additional information or contact your customer service representative. 

Datameer
Section
Info
titleINFO

Within these release notes Datameer X provides information and resolutions on improvements, bug fixes, and visual changes for 10.0.x maintenance release versions.  

Visit Datameer's Help Center for additional information or contact your customer service representative. 

Section

Datameer X 10.0.7

New Features

1

Supported Hadoop Distributions: Add Support for Cloudera CDP / Cloudera Runtime 7.0(.3)DAP-39597

Cloudera Runtime 7.0(.3) is now supported.


Improvements

1

Connection: Step 'Save' in connection edit mode - show a list of impacted artifacts when the authentication mode is set to 'SESSION_BASED'DAP-39087

A list of the impacted artifacts is now displayed when the authentication mode is set to 'SESSION_BASED' in the section 'Save' during connections.


2

Storybook: Enable new authentication dialog to request credentialsDAP-39088

In the storybook a new React component is enabled which requests either user 'Name', 'Password' or 'SSH Key' depending on the used connection.


3

Supported Hadoop Distributions: Add support for Hortonworks HDP 3.1.5DAP-39819

Datameer X now supports Hortonworks HDP 3.1.5.


4Tableau Export Connection: Parallel TDE/ Hyper file generation for Tableau Server exportsDAP-39846

Setting the property 'das.splitting.disable-combining=true' while configuring a Tableau connection runs the jobs significantly faster. It is verified that an export of a sheet to the Tableau server launches multiple tasks.
5Supported Hadoop Distributions: Add support for Cloudera CDH 6.3.3DAP-39933

Datameer X now supports Cloudera CDH 6.3.3.
6Import job/ DataLink Wizard: Request credentials while edit/ createDAP-39935

The property 'enable.authenticationOnJobRun=true' activates the session base authentication feature.

If a connection is configured as 'SESSION_BASED' and the authentication is already entered, the wizard will not show the ask for the credentials group.

If a connection switches from 'PROVIDED/TEMPORARY' credentials to 'SESSION BASED' and edit/ save the import job, a check after saving in the database table 'job_configuration_property' the properties 'datastore.user_name' and 'datastore.password' must be deleted/ not exist for a session based connection import/datalink job. Furthermore a check after saving in the database table 'dap_job_configuration' column 'schedule_enabled' must be true (1) for 'PROVIDED' and 'TEMPORARY' and false (0) for 'SESSION BASED'.


7Workbook: Display a "Save" button in toolbarDAP-40047

You can now click on a 'Save' button in the UI toolbar again to safe changes within a workbook.
8Workbook: Workbook Saving - Default to working pathDAP-40055

When closing a workbook for the first time, the workbook is saved in the folder it was created from and not in home folder upon exit. A workbook message does not read "saved under home folder" but using the actual path where the workbook is stored. When closing the File Browser the workbook is saved in the folder it was created from.
9Supported Hadoop Distributions: Add HBase 2 support for Cloudera CDH 6 versionsDAP-40109

Datameer X now supports HBase 2 support for Cloudera CDH 6 versions.
10Tableau Plug-in: Tableau Plug-in allow TDE/ Hyper exports to be setup from non-Linux systemsDAP-40147

The plug-in for Tableau now allows TDE/ Hyper exports to be setup from non-Linux systems.

Bug fixes

1

Column Inspector: 'Undefined' shown as a formula string in case of empty columnsDAP-39529

Empty columns do not show formulas in the column inspector any more.

(tick) Bug fixed.

2

Tableau Export: Tableau ExportJobs performance issueDAP-40034

The property 'das.splitting.disable-combining=true' while configuring a Tableau connection runs the jobs significantly faster and multiple tasks can be launched.

(tick) Bug fixed.

3

Workbook: Duplicating or copy/ pasting a workbook returns an errorDAP-40067

Workbooks can be copied and pasted without an error again.

(tick) Bug fixed.

4Workbook: 'EXPAND' function column position shows incorrect result for 'GROUPROWNUMBER'DAP-40087

Using the 'EXPAND(LIST)' function shows the correct sequence for 'GROUPROWNUMBER'.(tick) Bug fixed.
5Display key columns first flag causes 'IllegalStateException' in a 'Join Sheet'DAP-40121

Displaying the key columns first flag doesn't cause exceptions in a 'Join Sheet' any more.

(tick) Bug fixed.
6Workbook: Can't rename a column in the 'Join Sheet' because of a 'RegularRequirementNotMetFailure'DAP-40130

Column renaming in a 'Joint Sheet' is possible because the broken references check implemented for transformed sheets doesn't affect renaming at other sheet types.(tick) Bug fixed.
7Upgrading Datameer X 7.5.x to 10.1.2 workbook upgrade conversion breakageDAP-40140

Column changes are propagated to all depended workbooks and sheet, not only when upgrading.(tick) Bug fixed.
8Avro Import: 'NoSuchMethodError' with Cloudera CDH-5.14.4 (Hadoop distribution jackson jar binary incompatible)DAP-40142

Importing Avro files works on Cloudera CDH-5.14.4 distributions.(tick) Bug fixed.
9Workbook: Datameer X OOM crash when viewing sorted sheet dataDAP-40151

The new property 'das.data-download.merge-sort.max-total-file-size' is introduced to control (volume based) if the sorting will happen in full data view or not. The default value is 256MB.(tick) Bug fixed.
10Datameer X Mode: Prevent mixed content issues if Datameer X is configured in SSL modeDAP-40176

The button "New System" within 'External Systems' is available for entitled users such as an Admin in a SSL enabled installation, too.(tick) Bug fixed.



Section

Datameer X 10.0.6

New Feature

1

Add REST API call to look up what a UUID's properties areDAP-31229

For a UUID Datameer X provides the JSON containing the information about the artifact analog to a REST READ call for an artifact ID.


Improvements

1

Connection: In wizard section 'details' validate and store authentication on a session if the authentication session-based option is selectedDAP-39334

Authentication data is now validated and stored per session in the 'Connection Details' section within the connection wizard.


2

Connection: Show selected authentication mode on 'Connection Details'DAP-39335

Datameer X now shows the authentication mode on the 'Connection Details' page if the connection type provides such options (JDBC/MySQL, SSH, SFTP connections).


3

Connection: Implement a controller for session-based connection authenticationDAP-39936

A new service checks the session id and connection id for session-based authentication connections.


4Workbook: Enable flag for quick fix 'Workbook preview disappears in about 24 hours after execution'DAP-39928

The flag 'workbook-preview-service.has-new-data-check' is now added in the default properties and set 'true' per default.

Bug fixes

1

Workbook: Inconsistency in error messages for workbooks without a shared sourceDAP-38108

Error messages are adjusted and are now consistent for errors when working with a shared workbook without a shared source.

(tick) Bug fixed.

2

MySQL 8.x execute mysql-init.sql fails - no Grant options allowedDAP-39867

Datameer X has two new MySQL scripts for running MySQL 5.x and MySQL 8.x databases.

(tick) Bug fixed.

3

Security: After a failed login attempt, Datameer X exposes the LDAP query to fetch usersDAP-39953

The error message is shown again as expected and does not expose the LDAP query to fetch users anymore.

(tick) Bug fixed.

4Google Cloud Storage is bundled with the wrong Hadoop libraryDAP-39979

The Google Cloud Storage connector is now bundled with all Hadoop 3 versions.(tick) Bug fixed.
5Infographics: DataSetsController REST API copies full workbook or import job data to DFS (EhCache), therefore the conductor disk space is filling upDAP-39990

The DFSCache button on the developer's page is working again and data is now cleaned up as expected.(tick) Bug fixed.
6DevController: Cache Monitor 'Clear Cache' doesn't purge dataDAP-40005

The DFS cache is now cleared when clicking the "Clear Cache" button.(tick) Bug fixed.
7Workbook: Filtering on a column with the 'DENULLIFY' function causes an 'IllegalArgumentException'DAP-40017

The Datameer X function 'DENULLIFY' works as expected again.(tick) Bug fixed.
8Workbook: Show dependencies is very slow and keeps loadingDAP-40023

Due to performance, the polling interval while working with the file browser's dependencies view is now increased to 100 sec.(tick) Bug fixed.
9Workbook: Typo in backup and restore error messageDAP-40046

The error messages for duplicating and copying/pasting a shared workbook without sharing the data source is now correct.(tick) Bug fixed.



Section

Datameer X 10.0.5

Improvements

1

Add HBase 2.0 support for HDP 3.1.0 and HDP 3.1.4DAP-39823

Datameer X now supports importing HBase data for HDP 3.1.0 and 3.1.4 based distributions , both Both are shipped with HBase 2.0 support.


2

Add Support for Amazon EMR v5.28.1DAP-39877

Datameer X now supports Amazon EMR 5.28.1.


3

Add Support for Amazon EMR v5.29.0DAP-39878

Datameer X now supports Amazon EMR 5.29.0.


4

Remove 'superuser' related properties from 'live.properties'DAP-39910

The 'das.superuser.*' properties are now listed in the default properties only (disabled per default).


Bug fixes

1

Property API: Key-up Event for Property Info Input Fields of Property API Not Workingevent for property info input fields of property API not workingDAP-39015

A key-up event now gets recognized as a proper event and triggers the same actions as a general change event for input fields.

(tick) Bug fixed.

2

Hive Export: Drop & Create new tables doesn't work for partitioned tablesDAP-39390

Datameer X now validates Hive export definitions for partitioned tables and rejects those for 'Drop and create new table' as an unsupported operation.

(tick) Bug fixed.

3

Browser Search: Search results don't show correct results when changing the folderDAP-39556

Datameer X now updates the search results for folders if another folder from the file browser is selected.

(tick) Bug fixed.

4

Partition data table should be indexed for efficient group by queriesDAP-39662

To improve application performance and decrease load from Datameer's metadata database, indexes for 'data_partition.relative_path' and 'data_partition.index_key' have been added to the metadata database.

Upgrade scripts persist the changes in Datameer's metadata database accordingly.

(tick) Bug fixed.

5

Hive 3.1: Can Not Create Zookeeper not create ZooKeeper Hive JDBC connectionDAP-39947

Hive JDBC connection with Zookeeper ZooKeeper can now be created and beused for imports.

(tick) Bug fixed.



Section

Datameer X 10.0.4

Improvements

1

Workbook: SQL Editor should provide hint for all keyword typesDAP-39583

The autocomplete list now provides a hint if the selected keyword is a SQL statement, SQL function or SQL data type.


Bug fixes

1

Hive datalink to partitioned table with defer sampling shows an empty preview in the workbook DAP-38816

Including partition columns when editing a datalink shows a preview in a workbook.

(tick) Bug fixed.

2

Cannot export Runtime Analytics sheet to HiveDAP-39472

Runtime Analytic sheets can now beexported to Hive.

(tick) Bug fixed.

3

File Browser: Subfolders inside user's home folder also have home folder iconDAP-39524

Only the user's home folder has the home icon.

(tick) Bug fixed.

4

LDAP users with more than 1,500 members in their group are not recognizedDAP-39680

All users within a group ofmore than 1,500 members can now be authenticated.

(tick) Bug fixed.

5

Workbook: 'Sheet has no column' error on filtered, sorted, encoding or pivot sheets when renaming column in a source sheetDAP-39726

Renaming a column does not break the referenced sheet anymore. This affects all transform and data mining sheets.

(tick) Bug fixed.

6

'Server error' message in File Browser when selecting the user's home folderDAP-39767

The aforementioned error message isn't rendered anymore.

(tick) Bug fixed.

7

Import or export job run details page should not display 'Production Mode' informationDAP-39786

Import and export job run details pages don't render the 'Production Mode' information, as this is not relevant for these types.

(tick) Bug fixed.

8

Changing a formula will add an additional equals sign every time to the formula stringDAP-39830

Changing a formula doesn't add additional characters, like an equals (=) sign, anymore.

(tick) Bug fixed.

9

Plug-in Hive-JDBC: Include correct Hive JARs for EMR distributionsDAP-39843

The correct httpclient and httpcore JAR dependencies are now packaged with the plugin hive-jdbc for EMR distributions.

(tick) Bug fixed.



Section

Datameer X 10.0.3

Improvements

1

Add support for Hortonworks HDP 3.1.4DAP-39350

Datameer X now supports Hortonworks HDP 3.1.4.


Bug fixes

1

Hive 2 Plug-in: Show an error when list of databases or tables fails or is emptyDAP-38408

Datameer X shows a correspondingerror message in the user interface if tables are not available from the Hive Metastore database or cannot be derived.

(tick) Bug fixed.

2

Admin Page → Job Details Page: 'Show Logfile' shows log in the same windowDAP-39520

Clicking the 'Show Logfile' button now shows the log file in a new tab.

(tick) Bug fixed.

3

Open Data Format: Can't delete sync table for one sheet and set table with the same name for another sheetDAP-39525

Removing and adding a synchronization for another sheet with the samename is now possible in one step.

(tick) Bug fixed.

4

Workbook: Adding more than 5,000 characters string in the formula is always trimmed to 5,000 which causes the formula editor to highlight as errorDAP-39542

Datameer X now returns the non-trimmed value for the raw formula to be displayed in the formula editor.

(tick) Bug fixed.

5

Visual Explorer: Opening the Visual Explorer in a workbook which contains only a formula sheet leads to an errorDAP-39621

Visual Explorer now opens without an error.

(tick) Bug fixed.

6

'Migrating from one Hadoop cluster to another' link on admin Hadoop cluster page points to wrong version space

DAP-39734

The link now points to the correct version of the Datameer X product documentation.

(tick) Bug fixed.

7

Google Data Proc: (Un)secure hdfs tool doesn't workDAP-39757

The 'unsecure_hdfs_tool.sh' and 'secure_hdfs_tool.sh' scripts now work with Google Data Proc environments as well as with non-Google distributions.

(tick) Bug fixed.

8

WorkbookHealthCheck fails with java.lang.StackOverflowError exceptionDAP-39805

Datameer's WorkbookHealthCheck can now run for a large number of workbooks in an environment (e.g. 15,000 workbooks or more).

(tick) Bug fixed.



Section

Datameer X 10.0.2

Improvements

1

Tableau Hyper Export: correct libstdc++ library version loadingDAP-39115

Datameer X now injects the path to Tableau's native libraries into LD_LIBRARY_PATH for Tez environments when Tableau Server Datastores are used. This can be controlled via the custom property:

Code Block
languagetext
das.tableau.hyper.native-library-path-injection={true | false}



2

FileSplitter & InternalDatameerFileSplitter should use Hadoop FileSystem#listLocatedFileStatus for performance reasonsDAP-39516

Datameer X now shows improved performance when large numbers of files get imported via an import job or Datalink.

3

Add support for Cloudera CDH 6.3.2DAP-39657

Datameer X now supports Cloudera CDH 6.3.2.

4

Add support for Amazon EMR 5.28.0DAP-39695

Datameer X now supports Amazon EMR 5.28.0.

Bug fixes

1

Workbook: Context menu doesn't popup on double click on a value sometimesDAP-38301

The context menu is opened when a workbook record entry gets double-clicked.

(tick) Bug fixed.

2

JDBC connection fails due to hardcoded unicode propertyDAP-38958

The definition of 'useUnicode', 'characterEncoding' and 'charSet' as optional parameters for JDBC based connections is now allowed. The checkbox 'Enable Character Encoding' is now visible in the UI.

(tick) Bug fixed

3

Admin page: Plug-ins can not be paused or disabled in Firefox version 69.0DAP-39398

A plug-in now can be paused, deleted and restarted again when using Firefox version 69.0 as web browser.

(tick) Bug fixed

4

Workbook: Buttons are enabled in File Browser Dialog when they shouldn'tDAP-39495

Datameer X now ensures that the 'Add Data' button, 'Exchange Data' button and 'Open Workbook' are enabled in the correct context only.

(tick) Bug fixed

5

Workbook: When the 'Add Data' dialog is shown for a new workbook, it is not modalDAP-39510

The 'Add Data' dialog is now modal.

(tick) Bug fixed

6

Incorrect Daylight Saving Time (DST) related date value conversion during Hive importDAP-39569

Datameer X now properly handles DST related conversions while importing data values from Hive.

(tick) Bug fixed

7

HDFS BlockLocations topology paths lost when converting BlockLocation into FileBlockLocationDAP-39581

TopologyAwareAlgorithm now can optimize split combine operations depending on the data locality.

(tick) Bug fixed

8

Workbook: Renaming the source column of an Encoding sheet leads to an error if submitted by sheet switchDAP-39622

Renaming of a workbook column is handled when doing a sheet switch.

(tick) Bug fixed

9

Workbook: Jump to Column does not work correctlyDAP-39655

The correct workbook column position is recognized when using 'Jump to Column' dialog.

(tick) Bug fixed

10

Hardcoded Docu links should point to docu space configured in default.properties ('docu.url' )DAP-39693

Documentation links within Datameer X now point to the correct page within the Datameer X user documentation.

(tick) Bug fixed



Section

Datameer X 10.0.1

Improvements

1

Remember previous column position when switching sheets in a workbookDAP-22392

After switching sheets or opening other workbooks, Datameer X now switches back to the workbook sheet, row and column you were previously on.

2

File browser: show clickable links in dependency graphDAP-38886

When a node representing a Datameer X artifact is clickedin the dependency graph the file browser navigates to the folder in which the artifact is stored.

3

Improve record encoding for Datameer's Big Integer and Big Decimal to be Hive compatibleDAP-39367

Newly written and synced Parquet files are now compatible withand readable by Hive.

4

Housekeeping Service: encapsulate each Housekeeping task in its own exception handling.DAP-39506

Each Housekeeping task now encapsulates its own exception handling.

5

Optimize multimap value removal code in CombineOnlyDataLocalAlgorithm DAP-39565

Combining splits performance has been improved especially for a large number of splits to combine.

Bug fixes

1

Scheduled jobs remain indefinitely in the processing queue if the Kerberos password is changedDAP-39474

A scheduled job occurring between a Kerberos password change no longer blocks the processing queue.

(tick) Bug fixed

2

Hot deployment of a plug-in fails if the plug-in contains a DataStoreType implementationDAP-38885

A plug-in containing a DataStoreType can now be installed and uninstalled in the 'Admin' tab section, (e.g. Hive plug-in.)

(tick) Bug fixed

3

Deploy via Cloud Formation Template in AWS: Injected Examples aren't workingDAP-39160

Injected Examples are now backed by data sources in S3 and work for deployments on AWS EC2 instances as well.

(tick) Bug fixed

4

Housekeeping performance degradationDAP-39283

Improved robustness and traceability for housekeeping service allowing for the configuration of the number of HDFS artifacts to delete.

(tick) Bug fixed

5

REST API to download results doesn't return all records by setting 'rest.download-data.records-max=0'DAP-39386

Datameer X respects the setting 'rest.download-data.records-max=0' and provides all records for the download.

(tick) Bug fixed

6

Hive 2 plug-in: ImportJobs and Datalinks are failing on Hadoop Namenode and Resourcemanager high availability environmentsDAP-39397

Hive 2 plug-in based Importjobs and Datalinks now runs successfully on high availability environments.

(tick) Bug fixed

7

Workbook config: false errors on correct CRON patternDAP-39458

Datameer X now recognizes a valid CRON pattern and does not throw an error on a valid pattern.

(tick) Bug fixed

8

Visual Explorer should dismiss dialogs in order on 'Escape'DAP-39504

Pressing 'Escape' dismisses the respective front dialog (e.g. x-y-axes definition dialog instead of closing the whole Visual Explorer dialog). (CT- "Escape" use double quotes ")

(tick) Bug fixed