Importing from Google Cloud Storage

Owned by Juliane Wetzel
Last updated: Jul 13, 2020

INFO

Find here all information about importing data from Google Cloud Storage.

Requirements: Configuring Google Cloud Storage as a Connection

A Google Cloud Storage connection with Datameer X must be created before exporting data.

INFO

Learn about setting up a connection between Datameer X and Google Cloud Storage.

INFO

Note that a Google Cloud Storage bucket must not contain a "_" in its name.

Configuring Import Jobs with Google Cloud Storage

To import data from Google Cloud Storage:

  1. Click the "+" button and choose "Import Job" or right-click in the file browser and select "Create New" → "Import Job". The 'New Import Job' tab appears in the menu bar.
     
  2. Click "Select Connection". The dialog 'Select Connection' opens.
     
  3. Click on the connection for Google Cloud Storage and confirm with "Select". The connection is displayed.
     
  4. Select the required file type from the drop-down 'File Type' and confirm with "Next".
     
  5. Click "Browse" to select the folder/ file from the Google Cloud Storage. The 'Remote Data Browser' opens. 
     
  6. Select the required folder/ file and confirm with "Select". The data name is displayed in the 'File or Folder' field.
    TIP: You can scroll the results or filter for them. 
     
  7. Define the delimiter character the schema and the column names.
    INFO: The default value for delimiter is ','.
     
  8. If needed, choose a filter from the drop-down to allow the usage of a time range for applying time patterns in the file name path. 
  9. If needed, set file filter to exclude specific files in dependency to their age.
     
  10. If needed, modify advanced settings, e.g. the character encoding and confirm with "Next". The tab 'Data Fields' opens.
     
  11. Mark all required columns.
     
  12. If needed, view the raw records.
     
  13. If needed, enter placeholder values and confirm with "Apply".
     
  14. Decide how to handle invalid data. 
     
  15. Decide whether you want partition data.
     
  16. Confirm with "Next". The tab 'Schedule' opens.
     
  17. Decide whether the import is triggered manually or on a schedule.
     
  18. Select the way data is replaced or appended and confirm with "Next". The tab 'Save' opens.   
     
  19. If needed, enter an import job description.
     
  20. Mark the checkbox when the import shall start immediately after the saving. 
     
  21. If needed, enter the email address for several notifications and confirm with "Next". The 'Save Import Job' dialog opens.
     
  22. Select the path the data shall be imported, enter a name and confirm with "Save". Data Import from Google Cloud Storage is finished.