Secure File Uploads

Per default, file upload data is stored as plain text in HDFS. Administrators have the ability to add a compression codec in order to store those file uploads as non-human readable compressed files in HDFS. With the files compressed, Datameer users with required view/edit permissions can access, view, and edit the upload job while other users that don't have these job permissions would no longer have the ability to view the original plain text file directly from HDFS.

Configuring Hadoop to Compress File Uploads

1. Open Datameer as an administrator.
2. Click on the Admin tab at the top of the screen and select Hadoop Cluster from the side bar.
3. Click Edit to edit the Hadoop cluster.
4. Enter the following into the custom properties:
   • das.fileupload.originalfile.codec=<codec qualified class name>
   • io.compression.codecs=datameer.dap.common.util.ZipCodec,<codec qualified class name>
   • das.import.compression.known-file-suffixes=zip,gz,bz2,lzo,lzo_deflate,Z,<extension of user compression library>

   
   
5. Add the compression codec library jar in the folder <DATAMEER_HOME>/etc/custom-jars.
6. Re/start the Datameer application.