Supported Data Types
Format / System | Importing and exporting | Supported compression | Product tested version | Datameer version availability | Notes | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Import | Export | .gz | .bz2 | .lzo | Snappy | .zip | .Z | Import | Export | |||
| Amazon Redshift |  | | 8.2, 8.4 | 5.5 | 5.5 | Native Amazon Redshift JDBC 4.1 driver or a PostgreSQL jdbc driver can be used. | ||||||
| Apache Avro | | | | | | 2.1.4 | 2.1.4 | |||||
| Apache Web Server Logs | |  | ||||||||||
| Azure Blob Storage | | | 4.0 | 4.0 | (For HDP2.0, 2.2 and CDH+4 users) Contact Datameer services for info. | |||||||
| Cassandra | | | 0.6.5 | Public available plugin, see https://github.com/zznate/datameer-cassandra-plugin | ||||||||
| Cobol Copybooks | | | ||||||||||
| CSV/TSV, etc. | | | | | | |||||||
| DB2 | | | 8.1 (since 1.3.9), 9.7 Express-C | |||||||||
| Excel Workbooks | | | 2007 and after | 2.1.4 | 2.1.4 | |||||||
| Facebook Graph API (files) | | | ||||||||||
| Fixed Width Text | | | ||||||||||
| Google Spreadsheets | |
| 2.1 | |||||||||
| Greenplum | | | HD 1.1, HD 2.1, 4.1.1.1 | |||||||||
| HBase | | | 0.98.x, 0.96.x, 0.94.x, 0.92.x, 0.90.x | In order to satisfy the classloader requirements, hbase-protocol.jar must be included in Hadoop's classpath and the root Datameer classpath (/etc/custom-jars) for version 0.96.1 to 0.98.0
| ||||||||
| Hive Metastore | | | 0.13, 0.12, 0.11, 0.10, 0.9, 0.8, 0.7 | Export file format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | ||||||||
| HiveServer2 | | | 0.13, 0.14, 1.1.0.x | 5.9 | ||||||||
| HSQL-DB | | | ||||||||||
| HTML | | | 3.1 | |||||||||
| JSON | | | ||||||||||
| Log4j Log File | | | 2.1.5 | 2.1.5 | ||||||||
| MBOX (email archive files) | | | ||||||||||
| MS IIS Web Server Logs | | | ||||||||||
| MSSQL | | | SQL Express 2005 and 2008 | |||||||||
| MySQL | | | 5.5, 5.6, 5.7 | |||||||||
| Netezza | | | 6.0.6 | |||||||||
| Oracle | | | 10g XE, 11g XE | |||||||||
| OpenStack Swift | | | 5.11 | 5.11 | ||||||||
| ORC (Optimized Row Columnar) | | | | | | 4.5, 5.1 | Only supported in conjunction with Hive | |||||
| Parquet | | | | | | Parquet 2.1 | 5.3 | 5.3 | Parquet documentation | |||
| PostgreSQL | | | 9.0.x | |||||||||
| RCFile (Record Columnar File) | | | ||||||||||
| Sequence Files with Metadata | | | | | | |||||||
| Spark | | | 5.10 | 5.10 | ||||||||
| Sybase IQ | | | 12.7 | |||||||||
| Tableau | | | 5.7 | |||||||||
| Teradata | | | 12 & 13 | Teradata database needs to be configured to support the appropriate character set. | ||||||||
| Teradata Aster | | | 5.0 | |||||||||
| Text Files | | | | | | |||||||
| Twitter Firehose (files) | | | ||||||||||
Vertica |
|
| 5, 6.0, 6.1 | |||||||||
XML |
|
| | | | |||||||
Supported File Protocols
Protocol | Input | Output |
|---|---|---|
File |
|
|
HDFS |
|
|
SSH (SCP and SFTP) |
|
|
S3 |
|
|
Supported File compression codecs
Codec | Input | Output |
|---|---|---|
.gz |
|
|
.bz2 |
|
|
.lzo |
|
|
| Snappy | |
|
.zip |
| |
| .Z | | |
* LZO: tested with Fedora lzo.i386 v2.02-3.fc8 and http://github.com/kevinweil/hadoop-lzo (state from 2010-Jun-20)
Supported File Systems
Customer | File system | Special Hadoop configuration |
|---|---|---|
appistry | storage:/ | fs.storage.impl=org.apache.hadoop.fs.appistry.FabricStorageFileSystem |