...
...
...
...
| Info | ||
|---|---|---|
| ||
Note when using Hive 3: We are now converting all Datameer DATE columns to Hive TIMESTAMP and Hive DATE type data when exporting to HIVE 3 from the Datameer server time zone to UTC. And on import from Hive TIMESTAMP and Hive DATE back from UTC to Datameer server time zone. Never set the property "hive.parquet.timestamp.skip.conversion" to "TRUE". This would result in inconsistent data being stored in HIVE. |
| Table of Contents |
|---|
Configuring Hive Server2 as a Connection
...
There are options for configuring data field type mapping from Datameer to HiveServer2 upon export.
- In Datameer versions through 7.4, "Classic" is the default setting for the Hive plug-in. In Datameer versions 7.5+, Hive implicit mapping is used. Details of this mapping can be found under Exporting: Datameer Versions 7.5+ .
- Hive specific is similar to the default setting with the change that Datameer's BigDecimal is mapped to Hive's Decimal and Datameer's Date is mapped to Hive's Timestamp data type.
- BigDecimal
- New Table - Datameer exports the BigDecimal type to the Hive type Decimal with a precision (total number of digits) of 38 and a scale (number of digits to the right of the point) of 18.
- Existing Table - Datameer exports the BigDecimal type to the Hive type Decimal with the precision and scale defined on the Hive server.
- A maximum is set at a precision scale of (38,37).
- If an exported value doesn't fit within the precision/scale of either a new or existing table, a failure occurs.
- Date
- New Table - Datameer exports the Date type to the Hive type Timestamp.
- Existing Table - Datameer exports the Date type to the Hive type Timestamp/Date/String depending on what is defined on the Hive server.
- BigDecimal
...