File Path and File Name Patterns

Dynamic file path and file name patterns can be used when creating certain types of import and export jobs. If you are importing or exporting data to a file-based connection, you must specify a file type and name. If you set up your import/export job to update automatically on a specified schedule, then the new file overwrites the old one. This can be avoided by using dynamic file path and name patterns.

File Path Placeholders

The following variables are replaced by dynamic values during import / export:

* An auto increment number only works if the import / export job is split into more than one file. The first split is 1 and then continues to the total number of imported / exported files.

Example

/home/data/%user%/%year%/%month%/%day%/myFile.csv
is replaced with
/home/data/John/2009/02/08/myFile.csv
when e.g., an import job is executed on Feb. 8th, 2009 by the user John.

Custom File Path Placeholders

You can create custom placeholders if additional variables are required for the file path.

Create custom path placeholder:

1. Open the default.properties file.

2. Add in the custom property and save the file.

   system.property.envname.<placeholder name>=<folder name>

3. Open or restart Datameer

4. Create or open an import/export job.

5. Select a filesystem based connection (Example: SFTP, HDFS, FTP, ...) and file type.
6. Enter the file path using %env:<placeholder name>% as defined in the default.properties file.
7. Finish completing the import/export job wizard.
   If using Smart Execution:
8. Repeat steps 2-8 for the das-common.properties file.

Example

Saving the following to the default.properties folder:

system.property.envname.PRODUCT=MyProduct

The file path:
/home/data/path/%env:PRODUCT%/myFile.csv

is replaced with:
/home/data/path/MyProduct/myFile.csv

Date Patterns

Defining date patterns.

Examples

When no patterns are used in file names, existing files are replaced.

Other Tools Used to Write File Paths

Other than writing the direct file path, using default pattern placeholders, building your own placeholders, or using date patterns, Datameer X has other tools useful to write file paths.