| Table of Contents |
|---|
Supported Data Types
Format / System | Importing and exporting | |
|---|---|---|
Product tested version | Datameer version availability | Notes | |
|---|---|---|---|
| Import | Export | ||
| Import | Export | ||
|---|---|---|---|
| Amazon Redshift |  | | 8.2, 8.4 |
| Native Amazon Redshift JDBC 4.1 driver or a PostgreSQL jdbc driver can be used. | ||
| Apache Avro | | |
| Supports default compression types. | ||||||
| Apache Web Server Logs | |  | ||||
| Azure Blob Storage | | |
(For HDP2.0, 2.2 and CDH+4 users) Contact Datameer services for info. | ||||||
| Cassandra | | | 0.6.5 | Public available plugin, see https://github.com/zznate/datameer-cassandra-plugin | ||
| Cobol Copybooks | | | ||||
| CSV/TSV, etc. | | |
| Supports default compression types. | ||||||
| DB2 | | | 8.1 (since 1.3.9), 9.7 Express-C | |||
| Excel Workbooks | |
| 2007 and after |
| Fixed Width Text | | |
| Google BigQuery | |
| 7.5 | |||
| Google Spreadsheets | |
|
| Greenplum | | | HD 1.1, HD 2.1, 4.1.1.1 | |||
| HBase | | | 0.98.x, 0.96.x, 0.94.x, 0.92.x, 0.90.x | In order to satisfy the classloader requirements, hbase-protocol.jar must be included in Hadoop's classpath and the root Datameer classpath (/etc/custom-jars) for version 0.96.1 to 0.98.0
| ||
| Hive JDBC | | | ||||
| Hive Metastore | | | 0.13, 0.12, 0.11, 0.10, 0.9, 0.8, 0.7 | Export file format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | ||
| HiveServer2 | | | 0.13, 0.14, 1.1.0.x |
| HSQL-DB | | | ||||
| HTML | | |
| JSON | | | ||||
| Log4j Log File | | |
| MBOX (email archive files) | | | ||||
| MS IIS Web Server Logs | | | ||||
| MSSQL | | | SQL Express 2005 and 2008 | |||
| MySQL | | | 5.5, 5.6, 5.7 | |||
| Netezza | | | 6.0.6 | |||
| Oracle | | | 10g XE, 11g XE | |||
| OpenStack Swift | | |
| ORC (Optimized Row Columnar) | |
|
| 7.1 |
| |
| Parquet | | |
| Parquet 2.1 |
| ||||||
| PostgreSQL | | | 9.0.x | |||
| PowerBI | | | ||||
| RCFile (Record Columnar File) | |
| 7.1 | Export supported in conjunction with Hive only for existing partitioned Hive tables. | ||||
| Sequence Files with Metadata | | | Supports default compression types. | |||
| Snowflake | | |
| 7.1 | 7.1 | ||
| Spark | | |
| Sybase IQ | | | 12.7 | |||
| Tableau TDE | | |
| Tableau TDSX | | | ||||
| Tableau Hyper | | | 7.4 | Hadoop cluster native system library requirements:
| ||
| Teradata | | | 12 & 13 | Teradata database needs to be configured to support the appropriate character set. | ||
| Teradata Aster | | | 5.0 | |||
| Text Files | | |
| Supports default compression types. | ||||||
Vertica |
|
| 5, 6.0, 6.1 | |||
XML |
|
|
| Supports default compression types. |
Supported File Protocols
Protocol | Input | Output |
|---|---|---|
File |
|
|
HDFS |
|
|
SSH (SCP and SFTP) |
|
|
S3 |
|
|
| Note | ||
|---|---|---|
| ||
Datameer supports Bitverse SSH Server/Client for the Windows platform. The root paths to be specified while creating the connection should look something like: /c:/mydata/folder1 |
| Anchor | ||||
|---|---|---|---|---|
|
...
file compression codecs
Codec | Input | Output | Default compression | Notes |
|---|---|---|---|---|
.gz |
|
| ||
.bz2 |
|
| ||
.lzo |
|
| |
| Additional native libraries are required |
| . | ||||
| Snappy | |
| | |
.zip |
| | |
|
| .Z | | |
* LZO: tested with Fedora lzo.i386 v2.02-3.fc8 and and http://github.com/kevinweil/hadoop-lzo (state from 2010-Jun-20)
Supported File Systems
Customer | File system | Special Hadoop configuration |
|---|---|---|
appistry | storage:/ | fs.storage.impl=org.apache.hadoop.fs.appistry.FabricStorageFileSystem |