Behavior Change

INFO

Find behavior changes sorted by version here.

Release 10.1

Housekeeping

Subject

New housekeeping property

Behavior Change

Datameer X introduces the new property to configure the amount of deletion of file artifacts.

# Maximum number out-dated file artifacts that should be deleted from HDFS per housekeeping run
housekeeping.run.delete.outdated-file-artifacts=10000.

Versions Affected & Ticket Reference

10.0/ 10.1 - DAP-39283

Hive Mapping

Subject	Improve record encoding for Datameers big integer and big decimal to be Hive compatible
Behavior Change	Newly written and synced Parquet files are now compatible and readable by Hive.
Versions Affected & Ticket Reference	10.0/ 10.1 - DAP-39367

Column Encoding

Subject	Columns with very high cardinality have notification about one-hot-encoding
Behavior Change	Workbook columns with high cardinality now show a warning message in the encoding inspector.
Versions Affected & Ticket Reference	10.1 - DAP-39545

Release 10.0

Hive Plugin

Subject	Export data mapping changes
Behavior Change	While export into existing tables Datameer`s BigInteger can also be exported to Hive`s BigDecimal.
Versions Affected & Ticket Reference	10.0 - DAP-39147

Subject	Added support to export into new unpartitioned table with different Hive output format
Behavior Change	Datameer X now supports exports with selectable output format into a new unpartitioned table. You can select the output format in the export wizard.
Versions Affected & Ticket Reference	7.5/ 10.0 - DAP-38804

Parquet Import

Subject	Enabled support of Parquet files containing binary, fixed, int32 and int64 DECIMAL column types
Behavior Change	The import of Parquet files has now enabled supporting INT32 and INT64.
Versions Affected & Ticket Reference	7.4./ 7.5/ 10.0 - DAP 39110

Avro Import

Subject	Support FIXED, BYTES decimals for Avro file
Behavior Change	Datameer X supports now 'fixed with logical type decimal' and 'bytes with logical type decimal' on Avro file import.
Versions Affected & Ticket Reference	7.4/ 7.5/ 10.0 - DAP 39006

Workbook Info Dialog

Subject	Workbook Info dialog removed
Behavior Change	The 'Workbook Info' dialog is removed. The reference sheets and data source sheets have their reference shown in the corresponding sheet inspector now.
Versions Affected & Ticket Reference	10.0 - DAP-38689

Workbook Inspector

Subject	Enable list of formulas used by a formula sheet
Behavior Change	Since the 'Workbook Info' dialog is removed, you can find the list of formulas used by a formula sheet in the 'Sheet Inspector' in a collapsable panel now.
Versions Affected & Ticket Reference	10.0 - DAP-39102

Workbook Columns

Subject	Column names can be case sensitive
Behavior Change	Column names can now have the same names but different capitalisations, e.g. 'Foo', 'foo', 'FoO'.
Versions Affected & Ticket Reference	7.2/ 7.4/ 7.5 - DAP-38825

Subject	Document controllable Join column transpose
Behavior Change	You can now configure if join columns should be transposed to the front of the join sheet.
Versions Affected & Ticket Reference	10.0 - DAP-30214

Subject	Case-sensitive column names in workbook can be used in all sheet types
Behavior Change	All case-sensitive column names in a workbook can be used in all sheet types.
Versions Affected & Ticket Reference	7.5/ 10.0 - DAP-38825

Job Configuration

Subject	Take hadoop properties capability in account when overwriting configurations
Behavior Change	You can now overwrite the Hadoop properties if you have the capability to execute that. Datameer X provides a capability evaluation.
Versions Affected & Ticket Reference	7.4/ 7.5/ 10.0 - DAP-38342

New Property

Subject	Document usage of 'das.cluster.plugin.resources.fixer.enabled'
Behavior Change	A new property to mitigate YAN-3591 and YARN-6641 is introduced: 'das.cluster.plugin.resources.fixer.enabled'.
Versions Affected & Ticket Reference	10.0 - DAP-38032