Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 14 Next »

7.5.8 Release Notes

New Feature

1Add REST API call to look up what UUID's properties areDAP-31229

For a UUID Datameer provides the JSON containing the information about the artifact analog to a REST READ call for an artifact ID.

Improvements

1

Remove 'superuser' related properties from 'live.properties'DAP-39910

The properties 'das.superuser.enabled', 'das.superuser.username' and 'das.superuser.password' are now removed from the 'live.properties' and listed in the default properties only. The properties are disabled per default.

Bug Fixes

1

Workbook: Inconsistency in error messages for workbooks without shared sourceDAP-38108

Error messages are adjusted and are now consistent for error when working with a shared workbook without a shared source.

(tick) Bug fixed.

2

Infographics: DataSetsController REST API copies full workbook or import job data to DFS (EhCache), therefore the conductor disk space is filling upDAP-39990

The DFSCache button on the developer's page is working again and data is now cleaned up as expected.

(tick) Bug fixed.


7.5.7 Release Notes

Improvements

1

Add Support for Amazon EMR 5.28.1DAP-39877

Datameer now supports Amazon EMR 5.28.1.


2

Add Support for Amazon EMR 5.29.0DAP-39878

Datameer now supports Amazon EMR 5.29.0.

Bug Fixes

1

Property API: Keyup event for property info input fields of the Property API is not workingDAP-39015

Keyup event now triggers the same actions as for example a change event.

(tick) Bug fixed.

2

Cannot export Runtime Analytics sheets to HiveDAP-39472

Runtime Analytics sheets can now be exported to Hive.

(tick) Bug fixed.

3

Partition data table should be indexed for efficient group by queriesDAP-39662

Datameer now adds an index for 'data_partition.relative_path' and 'data_partition.index_key' columns to improve query performance to its metadata database.

(tick) Bug fixed.

4

LDAP users with more than 1,500 members in their group are not recognizedDAP-39680

All users within a group of more than 1,500 members can now be authenticated.

(tick) Bug fixed.

5

WorkbookHealthCheck fails with java.lang.StackOverflowErrorDAP-39805

Datameer's 'WorkbookHealthCheck' now successfully runs for a large number of workbooks in an environment (e.g. 15,000 workbooks or more).

(tick) Bug fixed.

6

Plug-in Hive-JDBC: Include Hive common JARs for Amazon EMR distributionsDAP-39843

The correct 'httpclient' and 'httpcore' JAR dependencies are now packaged with the plug-in hive-jdbc for Amazon EMR distributions.

(tick) Bug fixed.


7.5.6 Release Notes

Improvements

1

Tableau Hyper Export: correct libstdc++ library version loading DAP-39115

Datameer now injects the path to Tableau's native libraries into LD_LIBRARY_PATH for Tez environments when Tableau Server Datastores are used. This can be controlled via the custom property:

das.tableau.hyper.native-library-path-injection={true | false}

2

Add Support for Amazon EMR 5.28.0DAP-39695

Datameer now supports Amazon EMR 5.28.0.

Bug Fixes

1

Workbook: Buttons are enabled in File Browser Dialog when they shouldn'tDAP-39495

Datameer now ensures that the 'Add Data' button, 'Exchange Data' button and 'Open Workbook' are enabled in the correct context only.

(tick) Bug fixed.

2

Incorrect Daylight Saving Time (DST) related date value conversion during Hive importDAP-39569

Datameer now properly handles DST related conversions while importing data values from Hive.

(tick) Bug fixed.

3

Workbook: Renaming the source column of an Encoding sheet leads to an error if submitted by sheet switchDAP-39622

Renaming of a workbook column is handled when executing a sheet switch.

(tick) Bug fixed.

4

Plug-in Hive: Hive Export fails with wrong impersonating user when it is triggered by dependent workbook with a different ownerDAP-39754

Datameer now impersonates the correct user when an export job from a workbook is triggered.

(tick) Bug fixed.


7.5.5 Release Notes

Improvements

1

Optimize multimap value removal code in CombineOnlyDataLocalAlgorithm DAP-39565

Combining splits performance has been improved especially for a large number of splits to combine.


2

Add Support for CDH 6.3.2DAP-39657

Datameer now supports Cloudera CDH 6.3.2.

Bug Fixes

1

Hot deployment of a plug-in fails if the plug-in contains a DataStoreType implementationDAP-38885

A plug-in containing a DataStoreType can now be installed and uninstalled in the 'Admin' tab section, (e.g. Hive plug-in.)

(tick) Bug fixed.

2

REST API to download results doesn't return all records by setting 'rest.download-data.records-max=0'DAP-39386

Datameer respects the setting 'rest.download-data.records-max=0' and provides all records for the download.

(tick) Bug fixed.

3

Hive 2 plug-in: ImportJobs and Datalinks are failing on Hadoop Namenode and Resourcemanager high availability environmentsDAP-39397

Hive 2 plug-in based Importjobs and Datalinks now run successfully on high availability environments.

(tick) Bug fixed.

4

'Workbook' tab doesn't close after 'Close & Save' DAP-39417

After closing and saving a workbook, the 'Workbook' tab closes and the user is redirected to the 'File Browser' tab.

(tick) Bug fixed.

5

Hive String queryTemplate for SHOW PARTITIONS is incorrect DAP-39438

The string for queryTemplate now shows the required syntax using back-ticks for every single part of the Hive query.

(tick) Bug fixed.

6

Visual Explorer: Explorer can't be closed via 'Escape' keyDAP-39446

Visual Explorer dialog now closes when the 'Escape' key is pressed.

(tick) Bug fixed.

7

Workbook config: false errors on correct CRON patternDAP-39458

Datameer now recognizes a valid CRON pattern and does not throw an error on a valid pattern erroneously.

(tick) Bug fixed.

8

Scheduled jobs remain indefinitely in the processing queue if the Kerberos password is changedDAP-39474

A scheduled job occurring between a Kerberos password change no longer the processing queue.

(tick) Bug fixed.

9

Visual Explorer should dismiss dialogs in correct order on 'Escape'DAP-39504

Pressing 'Escape' dismisses the respective front dialog (e.g. x-y-axes definition dialog instead of closing the whole Visual Explorer dialog).

(tick) Bug fixed.

10

Workbook: save dialog doesn't open when user selects 'Close & Save'DAP-39509

Closing a new workbook from the file browser now redirects to the workbook save dialog.

(tick) Bug fixed.


7.5.4 Release Notes

New Feature

1

Support Engineer Plug-in - Customer Usage Analytics

DAP-36640


The SEP contains both high-level metadata and various logs from within Datameer environments. This allows administrators to easily pass necessary information to the Datameer support engineers.


Improvements

1

Plug-in Hive: Support Export into new table in format Parquet, ORC, Avro or Text File

DAP-38804


When exporting data to Hive, the user is presented with a simple drop-down menu to select the file format. You can choose between Parquet, ORC, Avro and Text File.

2

Improve searching for folders within the file browser

DAP-38862


Easily search for files by paths or by folder names.

3

Support for Parquet File Format with types Date and Timestamp

DAP-39000


If date and timestamp are data types, use monospace. The data types DATE and TIMESTAMP are supported when exporting to Parquet file type.


4Materialize all SQL sheet inputs, join optimizerDAP-39011

Execution of SQL joins within workbooks is decreased when using the improved join optimizer.


5

Workbook validity report

DAP-39389


Datameer provides report that identifies workbooks missing data sources or other similar issues.

6

Hive: partition location cache scheduling has been removed

DAP-39392


Datameer does not cache partition locations on a schedule, only on demand.

7

Production Mode: In-Place Filter Prevents Projection Push-down for Data Links

DAP-39395


In production mode, the column projection feature is available for data links. This is a performance improvement for workbooks using filter conditions and not processing all columns ingested from the datasource.

8

Add Support for Amazon EMR v5.27.0

DAP-39457


Datameer now supports Amazon EMR 5.27.0.

9

Add Support for CDH 6.2.1

DAP-39461


Datameer now supports CDH 6.2.1.

10

Add Support for CDH 6.3.1

DAP-39462


Datameer now supports CDH 6.3.1.

Bug Fixes

1

Artifact ( IJ, Wbk, FU, DL, EJ ) status not updated in dependency view

DAP-37822


The entity status in the dependency view reflects the current status (success/ error/ canceled) of this entity.

(tick) Bug fixed.

2

Workbook: Valid SQL queries are not highlighted as errors

DAP-37984


Datameer erroneously marked valid SQL statements as false.

(tick) Bug fixed.

3

Download workbook data runs in ArrayIndexOutOfBoundsException

DAP-38839


The download of workbook data is not influenced by workbook changes after the run. Furthermore partition selection is no longer possible after the sheet containing a date column was deleted.

(tick) Bug fixed.

4

Error when importing a worksheet from a sorted workbook in production mode

DAP-38874


Importing a worksheet from a sorted workbook in production modes has been fixed.

(tick) Bug fixed.

5

Hive2-plugin: Append-Export in s3 located table overwrites existing data ( unpartitioned tables )

DAP-38918


Records in a Hive table with S3 location are appended when the job export configuration to this table is configured to "append" records. When the configuration is set to "overwrite", the records are overwritten.

(tick) Bug fixed.

6

AdminVariables: Hover state of buttons has been corrected

DAP-39113


While editing admin variables within IE 11, all buttons have the correct color now. The hover state disappears as expected.

(tick) Bug fixed.

7

Error when creating a variable with more than 255 chars

DAP-39114


Variable names can have a maximum of 255 characters. Variable values can have a maximum of 255 characters.

(tick) Bug fixed.

8

Workbook: variables not listed on workbook details / job run details page when job was scheduled/data driven

DAP-39154


When a workbook is data driven by an import job or scheduled, all variables are listed within the job run details/ details page.

(tick) Bug fixed.

9

SER: improve how the config files are displayed

DAP-39211


JSON config file are displayed as key-value pairs, making them both machine and human readable.

(tick) Bug fixed.

10

Workbook: formulas are applied on loss of focus

DAP-39258


When a column looses focus or when adding a new sheet, any changes to a formula are applied without user interaction.

(tick) Bug fixed.

11

Backup/Restore does not abort when encountering broken artifacts

DAP-39297


Even if there are broken artifacts, a backup continues and logs the broken files.

(tick) Bug fixed.

12

Hive-Jdbc: Rescan Schema leads to SaslException on NMU environment

DAP-39308


Rescan schema now works without generating an exception for hive JDBC running in DM NMU mode

(tick) Bug fixed.

13

Hive2-plugin: Export with mode Create New and policy RECREATE fails if table exists on NMU environment

DAP-39318


The export with mode "Create new if not exist" and policy "Drop and create new table" (RECREATE) works if the table exists on the NMU environment. This works for external and managed tables as well for HiveServer2 and Metastore based export jobs.

(tick) Bug fixed.

14

Workbook: inconsistent column names if sheet has empty columns

DAP-39322


Column naming for SQL sheets now follows the usual conventions (A, C, C1, ...).

(tick) Bug fixed.

15

Hive2-plugin: "Table Output Format" Select looks unformatted in Export Wizard

DAP-39324


The "Table Output Format" select field to the "Data Details" section within the Hive export wizard is now correctly formatted.

(tick) Bug fixed.

16

Admin page: inform user if SER plugin is disabled/unavailable

DAP-39326


When the Support Engineer Plugin is disabled or unavailable, the section "Support Engineer Plugin" is removed from the admin section.

(tick) Bug fixed.

17

SER: browser tab has wrong title

DAP-39328


Being on the "System & Usage Report" section now displays the correct heading ("System & Usage Report") in the browser.

(tick) Bug fixed.

18

SER: download icons disappears after clicking it

DAP-39329


After clicking the download icon for a log, the icon stays visible.

(tick) Bug fixed.

19

SER: environment information json is empty after download

DAP-39331


When downloading the whole "System & Usage" report, the environment jSON file contains content.

(tick) Bug fixed.

20

IE11: System & Usage Report shows blank page

DAP-39343


When opening the "System & Usage Report" section, all expected content appears. The download works again and single logs can be downloaded as expected.

(tick) Bug fixed.

21

Customer Upgrade: ExportJob forcing append

DAP-39365


Hive export jobs are no longer duplicated after upgrading from Datameer 7.1 to 7.5. In case of recreating or overwriting, old data from a Hive managed table is deleted. In case of recreating or overwriting from a Hive external table will not be deleted.

(tick) Bug fixed.

22

Error dialog has broken layout if variable name is too long

DAP-39393


Error dialogs with long variable names are displayed correctly. Long variable names are broken to the next line now.

(tick) Bug fixed.

23

Filebrowser: search function in folders with whitespace not working

DAP-39411


Searching on the file browser takes whitespace into account and doesn't cause errors.

(tick) Bug fixed.

24

Add to Formula doesn't add formula to formula editor from Browse Columns if working/browsing same sheet

DAP-39439


Clicking the "Add to formula" button adds a formula to the formulas editor from "Browse Columns" if working the same sheet.

(tick) Bug fixed.

25

v7.5: Customer Upgrade: Join Columns in Error

DAP-39456


Cascading errors in join sheets don't appear any more. After an upgrade to Datameer version 7.5, columns are migrated successfully. Circular dependencies are detected. Broken workbooks stay broken with the same error.

(tick) Bug fixed.


7.5.3 Release Notes

Improvements

1

Add support for Google Cloud Service (GCS) protocol to import and export files

DAP-20331


Datameer supports import to and export from39146 Google Cloud Service (GCS) connections.

2

Join Operation Should Compute Inputs in Parallel instead of one at a time

DAP-39146


For improved performance, join of two data links now computes inputs in parallel to prevent the creation of too many DAG(s) on the cluster, and the Map-Side join picks the smallest of the two inputs to cache in memory.

3

Plug-in Hive: improve implicit mapping for type BigIntegerDAP-39147

When exporting into existing Hive tables Datameer's "BIG_INTEGER" can also be exported as Hive "decimal."


4

Add Support for CDH 5.16.2

DAP-39131


Datameer now supports CDH 5.16.2.

5

Add Support for CDH 6.3.0

DAP-39169


Datameer now supports CDH 6.3.0.

6

Add Support for Amazon EMR 5.24.1

DAP-39132


Datameer now supports Amazon EMR 5.24.1.

7

Add Support for Amazon EMR 5.25.0

DAP-39170


Datameer now supports Amazon EMR 5.25.0

8

Add Support for Amazon EMR 5.26.0

DAP-39284


Datameer now supports Amazon EMR 5.26.0

Bug Fixes

1

Upgrade scripts are ignored when using Maria DB


DAP-39293


For a fresh installation, the optional_upgrade table in Datameer's metadata database now gets filled, therefore all included optional upgrades for the current Datameer version are set as expected.


(tick) Bug fixed.

2

BigQuery import job Data Details wizard page UI reload doesnt adhere to selected table


DAP-39230


When editing an existing BigQuery import, the selected table is the same as before the point of creation.


(tick) Bug fixed.

3

Google BigQuery import connector non-US region datasets failure


DAP-39217


BigQuery connector works for datasets residing in regions other than USA.
Note: the temporary dataset and dataset from which you are importing data must reside in the same geographical region.


(tick) Bug fixed.

4

After opening a workbook by path can't switch tabs back to that workbook


DAP-39197


It is possible to switch tabs from file browser to the workbook when opening a workbook by path.


(tick) Bug fixed.

5

Can't create a Variable if Datameer serves via SSL


DAP-39167


The mentioned source gets called via HTTPS as well when SSL/TLS is enabled in Datameer.


(tick) Bug fixed.

6

TDE/Hyper export with UNICODE symbols over U+D7FF fails job/crashes Datameer (in local ef)


DAP-39164


Characters that caused a crash when exporting a sheet to Tableau are replaced with HTML entities. Such replacements are logged in the application log.


(tick) Bug fixed.

7

Workbook: variables in formula editor are highlighted in red


DAP-39155


Defined variables are no longer highlighted as an error in workbook sheet formulas or SQL queries.


(tick) Bug fixed.

8

Workbook: error message is shown after duplication of a Recommendation or Column Dependencies sheet


DAP-39104


No error message is shown after duplication of a Recommendation or Column Dependencies sheet.


(tick) Bug fixed.

9

Workbook: editor for SQL query not working correct in some cases


DAP-39062


The SQL query editor now handles <space> and <return> key strokes as well as working with auto completion in the correct way.


(tick) Bug fixed.

10

Datameer is missing the  "X-XSS-Protection" HTTP response header


DAP-39048


`X-XSS-Protection: 1; mode=block` is set in the header of every response coming from the Datameer backend.


(tick) Bug fixed.

11

Workbook: show correct status bar and error/info dialog if workbook has status "timed out"


DAP-39044


Datameer now shows the correct status bar and error or information dialog when a workbook has been timed out.


(tick) Bug fixed.

12

Can't read DECIMAL from an Avro file


DAP-39006


Datameer now imports Avro data with primitive type bytes logicaltype decimal as BIG_DECIMAL in Datameer.


(tick) Bug fixed.

13

Workbook - Datasource Sheet: a schema change or an encoding change of the referenced import job breaks loading of the referencing workbook


DAP-38848


Datameer now opens the referencing workbook after a schema or encoding change to the referenced import job.


(tick) Bug fixed.

14

Workbook: Data link with workbook pushdown filter reported total records are not always transparent


DAP-37857


Datameer now distinguishes more clearly between availability of metrics for total and sample records. (Instead of displaying a count of total rows for a Datalink used by a workbook with a filter condition, we show the message "Showing <number> rows of an unknown number of total rows".


(tick) Bug fixed.


7.5.2.1 Release Notes

Bug Fixes

1

Can't create a Variable in the Admin section if Datameer serves via SSLDAP-39167

The mentioned source gets called via HTTPS as well when SSL/TLS is enabled in Datameer.



7.5.2 Release Notes

New Feature

1

Certify Hive Plugin for Amazon's EMR Distribution 5.24 to the Standard Datameer ApplicationDAP-38274

Improved connector to allow Hive access on EMR.

Improvements

1

Improve responsiveness of UI for customers with large number of job executionsDAP-39030

Querying very large tables for job executions could take seconds, rendering the file browser UI unusable. Datameer now has an index for the job_status column in the metadata database to speed information queuing and display.

2

Integrate Plug-in Code into Plug-in NMU to notify users by email when jobs fail due to missing/stale keytabDAP-38761

Users get now notified by email when a Datameer job is failing because of a missing or stale keytab file.

3

Workbook Variables: improve icon and edit behaviourDAP-38879

Datameer now switches to edit mode for workbook variables and renders buttons active/inactive according to their expected states.

4

No plugin with id '<plugin_name>' found for deserializing class '<plugin_internal_class>'DAP-38023

Datameer now handles occurrences of YARN-359 or YARN-6641 gracefully so that errors indicating a specific plug-in isn't available can be mitigated. The workaround can be activated with the das.cluster.plugin.resources.fixer.enabled property.

5

Pivot Workbook Inspector: do not show red border and 'Select values to include" if column was removed manuallyDAP-38910

Datameer now renders an empty column selection field for column names that do not exist.

Bug Fixes

1

Hive Plug-in - Export: Data-driven Hive ExportJobs submitting with random impersonated user
DAP-39046

Datameer now runs the Hive export job with correct user impersonation.

(tick) Bug fixed.

2

Datameer can't recognise Int32 and Int64 Decimal values on a Parquet File Ingestion
DAP-39110

Datameer now can handle Parquet files containing Binary, Fixed, INT32 and INT64, as well as DECIMAL column types


(tick) Bug fixed.

3

Workbook can not update SQL sheet when variable was deleted
DAP-39014

Datameer now allows to update a SQL Sheet with a valid query, even if former referenced workbook variable doesn't exist anymore.

(tick) Bug fixed.

4

Workbook: Pivot inspector - no hint when column is choosen but no value was selected
DAP-39128

Datameer renders a red border and indicates an error, if a column was choosen but no value was selected.

(tick) Bug fixed.

5

RESTv2 call to stop a job responds with error
DAP-38985

The REST call responds now with status 200 and stops the referenced job.

(tick) Bug fixed.

6

After re-run on a partitioned DataLink, partition information is missing in the workbook
DAP-38271

Datameer renders the partition information for a datalink in the Sunburst for selection, even if the datalink was triggered in between again.

(tick) Bug fixed.

7

Hive Plug-in: Transport Mode HTTP with Table Filter set to "Table and View" leads to an error on Kerberos secured environments
DAP-38979

This setup and configuration combination no longer generates an error.

(tick) Bug fixed.

8

Hive Plug-in: Hive Metastore - Create Delegation Token for Kerberos Secured Hive independent of configured Impersonation Mode
DAP-39034

Datameer now creates the required delegation token for accessing a Kerberos Secured Hive.


(tick) Bug fixed.

9

Pivot Sheet: Closing the error dialog with Enter or Escape key doesn't work
DAP-39008

Dialogs for Pivot Sheet operations now handle Enter and Escape key submissions in the expected way.

(tick) Bug fixed.

10

NPE when configure plug-in where preferences extension are disabled
DAP-39091

Datameer now correctly handles disabling an extension point of plug-in and does not produce a "NullPointerError".

(tick) Bug fixed.

11

Upgrade: Many Workbooks failing after upgrading to 7.4.6 because of wrong auto-resource computation
DAP-39055

Datameer auto-resource computation for required vCores no longer results in zero.


(tick) Bug fixed.

12

Undo/Redo throws exception after reorder SQL sheet with variable
DAP-38882

Undo/Redo now works without errors when re-ordering SQL sheets with variables.


(tick) Bug fixed.

13

Workbook: invalid CSRF token shows error message over actual warning
DAP-39026

Datameer only shows the warning message for the invalid session.

(tick) Bug fixed.

14

Wrong icons on workbook
DAP-39105

Datameer now renders the correct icons in the workbook.

(tick) Bug fixed.

15

Add new "Timed out" icon for jobs
DAP-38769

Datameer renders the time-out icon on job run details and job history page.

(tick) Bug fixed.

16

Workbook: Sunburst does not open for second sheet when adding data of 2 workbooks
DAP-38284

Datameer renders the Sunburst for partition selection for every partitioned data source of a workbook.

(tick) Bug fixed.

17

Workbook exchange data source should respect "kept" attribute of the datasource being replaced
DAP-39001

When exchanging a datasource, the "kept" flag respects the original data sheet "kept" value.

(tick) Bug fixed.

18

Workbook: wrong syntax highlighting on formula sheets
DAP-39130

Datameer now renders with the correct syntax highlighting in workbook formulas.


(tick) Bug fixed.

19

Cross-Site Scripting Vulnerability in Advanced Filter Dialog and SQL Sheet Query input fields
DAP-39149

Advanced filter dialog and SQL sheet query input fields are not vulnerable to Cross Site Scripting attempts.


(tick) Bug fixed.

20

Memory-Backed Map-Side Join will never be chosen when input is two Data Links
DAP-39095

The chosen join algorithm will be MEMORY_BACK_MAP_SIDE when two data links are being joined and at least one of the sides is small.


(tick) Bug fixed.

  • No labels