Uploading Files

Owned by Charles Bishop (Deactivated)
Last updated: Oct 14, 2019 by Collin Turner (Deactivated)

Use a file upload to bring data from a file on your desktop into Datameer on a one time basis. That data is then stored in HDFS with an explicit path in its configuration. For a specific single file, you can't upload a newer version. If you want to bring in a batch of files or schedule regular imports to update your data, consider using an import job instead.

You can upload files such as CSV files or any of the other supported file types including compressed (ZIP) files.

Creating a File Upload

To upload files:

  1. Click the + (plus) button and then File Upload or right-click in the navigation bar and select Create new > File Upload.
  2. Click Browse and navigate to the file you want to upload and click Open.
  3. Select the file type from the list and click Next.

  4. Enter the data details if needed and click Advanced to enter additional options. The choices vary depending upon the file type you selected. 

    Obfuscation is available with Datameer's Advanced Governance module.

  5. In the Encryption section, select which columns to obfuscate by entering the column names with a space between them. The data of columns you obfuscate is never pulled into Datameer. Click Next.

  6. If there were errors, you can view the log file or a sample of the raw data to help resolve the problem. Specify whether empty fields are accepted. The Empty value placeholders section gives you the ability to assign specific values as NULL. Values added here aren't imported into Datameer.
  7. Specify how to handle invalid records and click Next.
  8. Enter a description and click Save As.
  9. Name the file and click Save.

Editing an Uploaded File

To edit an uploaded file:

  1. Click the File Browser tab. 
  2. Click File Uploads from the navigation bar on the left side of the screen.
  3. Right-click the uploaded file you want to edit and click Configure. (As of v6.3, click Open.)
  4. Make your changes and click Next to move through the wizard.
  5. Click Save when you are finished.

Creating a Copy of an Uploaded File

To create a copy of an existing uploaded file:

  1. Click the File Browser tab. 
  2. Click File Uploads from the navigation bar on the left side of the screen.
  3. Right-click the uploaded file you want to copy.
  4. Click Duplicate.

The copy is created and is named "copy of " and the name of the original uploaded file.

Deleting an Uploaded File

The delete feature deletes the file upload in Datameer but doesn't delete the actual data.

To delete an uploaded file:

  1. Click the File Browser tab. 
  2. Click File Uploads from the navigation bar on the left side of the screen.
  3. Right-click the uploaded file you want to delete and click Delete.
  4. Click OK and confirm the deletion.

Linking Data to a New Workbook

You can link data to a new workbook.

To link data to a new workbook:

  1. Click the File Browser tab. 
  2. Click File Uploads from the navigation bar on the left side of the screen.
  3. Double-click the uploaded file name to link into a new workbook.
  4. Click Link Data in New Workbook.
  5. The data is loaded into a new workbook.

File Upload Size Limits

There is no size limit for a file upload though large file uploads might lead to performance problems. The maximum supported file upload size is 500 MB. 

Viewing the Uploaded File Size and Monthly Upload Sizes

You can view the count of processed bytes for each upload and their total volume counting towards the license term.

To view the processed bytes per single job execution and totals for that job configuration of uploaded files:

  1. Click the File Browser tab. 
  2. Click File Uploads from the navigation bar on the left side of the screen.
  3. The size of last job run is displayed first and the total for that job configuration is displayed in the next column.

If a new license term starts and the file upload is processed again, the count starts with a new total processed data amount.

Identifying Workbooks Affected by a File Upload Schema Change

Datameer gives you a notice when editing a file upload if a schema change affects a corresponding workbook.

To view which workbooks are affected by a schema change from the uploaded data:

  1. Complete the file upload configuration until reaching the Save section.
  2. Review the note box detailing the changes that this configuration save has on the previous save and which workbooks are affected.
  3. Check the box to email users of the workbooks that are affected by the new schema changes.