Upgrading Datameer

Upgrading Datameer

General Information

INFO

If you have a completely new Datameer X installation, upgrading Datameer X is not necessary.

The following is recommended before the upgrade is executed:

  • Make a plan for the application downtime scheduling, notifications and create a maintenance window for Datameer.

  • A database backup will be performed by the Upgrade Tool automatically. Locally stored Datameer X data must be copied in the 'das-data' folder - the HDFS data remains intact.

  • Since it is recommended to not copy old configuration files or scripts to the new location, note the changes you have made in the previous setup. 
    INFO: You will need this information to make necessary changes in the new configuration files.

  • A property file exists so Job Scheduler and Event Bus settings adjusted in the UI are read during Datameer X start up.

    • The property file "overrides.properties" is not written on Datameer X but your systems HOME folder. (Path = <home>/.datameer/overrides.properties)

    • This property file is auto-created when adjusting the Event Bus or Job Scheduler settings under the 'Admin' tab. 
      INFO: This file can also be manually added and edited.

    • This is the last properties file read on start up. (E.g., it overrides other property files like default.properties and deployMode.properties) 

    • This file can be modified to allow for storing and the restoring of custom properties.

Requirements

  • System requirements must be fulfilled.

  • The required Datameer X version must have been downloaded. Please get in touch with Datameer Support or your Datameer representative to receive the installation package download link.

  • Make sure you have at least Datameer X version 10.1 before upgrading to Datameer X version 11.

  • A compatible SQL connector jar file must be provided in the file package: '/etc/custom-jars'.

Upgrading Preparation

Upgrading From an Older Datameer X Release

INFO

Sometimes it is necessary to update in several release version steps, e.g. from Datameer. Updating from Datameer X versions lower than version 10.1 must first be upgraded to the latest Datameer X version 10.1.x. Upgrading directly to Datameer X version 11 is only from version 10.1 possible.

Updating the Installation Symlink

TIP

Symlinks that were created through the Datameer X installation must be updated through the updating process.

To update the symlink:

  1. Remove the existing symlink:

    rm current
  2. Create a new symlink and change the working directory:

    ln -s Datameer-<package> current cd current

Disabling Housekeeping/ Compaction Services

INFO

Housekeeping and Compaction services must be disabled to allow for a possible roll back and to avoid any potential data loss.

To disable the Housekeeping and Compaction services:

  1. Open the file ' /<Datameer X new version installation directory>/conf/default.properties'.

  2. Disable the Housekeeping Service by setting the property 'housekeeping.enabled to 'housekeeping.enabled=false'.

  3. Disable the Compaction Service by setting the property 'auto-compaction.enabled' to 'auto-compaction=false'.

  4. Restart Datameer X to apply the changes. 

Migrating Workbooks and Other Artifacts

INFO

Workbooks and other JSON files downloaded as a backup in older versions of Datameer X are not supported in newer versions of Datameer.

When upgrading to a newer version of Datameer X ensure that all needed workbooks and files are part of the migration process so that they can be used in your new version of Datameer.

Processing Workbook Validation Before Upgrading Datameer

INFO

Execute a Workbook Health Check to detect broken workbooks and review a workbook's structure and report any logical issues.

The Workbook Health Check is available as of 11.0.0 for Datameer X 11.

Backing Up Keyfiles and Keystore

If you have set up password encryption and/or enabled SSL  with custom certificates , backup your Keystore and Keyczar keyfiles.

Upgrading Process

INFO

The upgrading process consists of two parts:

  • upgrading the Datameer X application

  • upgrading the database

Upgrading the Application

INFO

Updating Datameer X using user/ group 'root' is recommended to change the permissions to the user/ group 'datameer' for security reasons.

INFO

View current running or queued jobs and pause the job scheduler so that jobs are not submitted to the cluster.

INFO

INFO: Do not copy the entire old 'conf/' directory or 'conductor.sh' script to the new location. You only need to make changes in the new file for the changes you made previously.