Upgrading Datameer

General Information

The following is recommended before the upgrade is executed:

• Make a plan for the application downtime scheduling, notifications and create a maintenance window for Datameer.
• A database backup will be performed by the Upgrade Tool automatically. Locally stored Datameer X data must be copied in the 'das-data' folder - the HDFS data remains intact.
• Since it is recommended to not copy old configuration files or scripts to the new location, note the changes you have made in the previous setup. 
  INFO: You will need this information to make necessary changes in the new configuration files.
• A property file exists so Job Scheduler and Event Bus settings adjusted in the UI are read during Datameer X start up.
  • The property file "overrides.properties" is not written on Datameer X but your systems HOME folder. (Path = <home>/.datameer/overrides.properties)
  • This property file is auto-created when adjusting the Event Bus or Job Scheduler settings under the 'Admin' tab. 
    INFO: This file can also be manually added and edited.
  • This is the last properties file read on start up. (E.g., it overrides other property files like default.properties and deployMode.properties) 
  • This file can be modified to allow for storing and the restoring of custom properties.

Requirements

• System requirements must be fulfilled.
• The required Datameer X version must have been downloaded. Please get in touch with Datameer Support or your Datameer representative to receive the installation package download link.

• Make sure you have at least Datameer X version 10.1 before upgrading to Datameer X version 11.

• A compatible SQL connector jar file must be provided in the file package: '/etc/custom-jars'.

Upgrading Preparation

Upgrading From an Older Datameer X Release

Updating the Installation Symlink

To update the symlink:

1. Remove the existing symlink:

   rm current

2. Create a new symlink and change the working directory:

   ln -s Datameer-<package> current
   cd current

Disabling Housekeeping/ Compaction Services

To disable the Housekeeping and Compaction services:

1. Open the file ' /<Datameer X new version installation directory>/conf/default.properties'.
2. Disable the Housekeeping Service by setting the property 'housekeeping.enabled to 'housekeeping.enabled=false'.
3. Disable the Compaction Service by setting the property 'auto-compaction.enabled' to 'auto--compaction=false'.
4. Restart Datameer X to apply the changes. 

Migrating Workbooks and Other Artifacts

Processing Workbook Validation Before Upgrading Datameer

Backing Up Keyfiles and Keystore

If you have set up password encryption and/or enabled SSL  with custom certificates , backup your Keystore and Keyczar keyfiles.

Upgrading Process

Upgrading the Application

To upgrade the application:

1. Stop the Datameer X application gracefully.

   <Datameer X Application Folder>/bin/conductor.sh stop

2. Unzip the upgrade file and move the MySQL JDBC driver located in <Datameer X path>/das-data/jdbc-jars to the new installation's <New Datameer X path>/das-data/jdbc-jars> folder.

   export DAS_DEPLOY_MODE=live

3. When upgrading in Workgroup or Enterprise versions, consider allocating additional memory in 'etc/das-env.sh'. Make the necessary changes to the 'etc/das-env.sh' file. 
   INFO: Memory requirements and the main JAVA options have not changed in Datameer X version 10., so the values introduced in the previous version could be re-used. 

   # Adjust max available memory (-Xmx) according to your needs
   # WARNING: Path variables that may contain blanks should be added to jetty.sh start_das() method (see DAP-6342)
   export JAVA_OPTIONS="-Xmx2048m -XX:MaxPermSize=384m -Xms256m -XX:MaxNewSize=448m -XX:SurvivorRatio=6 -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=80 -XX:+HeapDumpOnOutOfMemoryError -XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled" 
   export JAVA_OPTIONS="$JAVA_OPTIONS -Dfile.encoding=utf-8"

4. If needed, change the container sizing of the Map, Reduce and AM containers which respectively correspond to MapReduce and Tez jobs. Therefore change the settings of the properties:

   das.job.map-task.memory=2048
   das.job.reduce-task.memory=2048
   das.job.application-manager.memory=2048

5. If existing, copy the files from the 'das-data' folder of the old distribution to the new location. 
   INFO: Keep the original 'das-data' information as a backup. 

   cp -r /<old-location>/das-data /<new-location>/

6. If needed, update the folder in which plug-in configurations are stored. Therefore open the 'default.properties' file from the file system of the previous Datameer X version, search for 'system.property.plugin.configs.dir=<folder name>' and open the 'default.properties' file from the file system of the Datameer X upgrade version. Now check or update the new property value to be the same folder name in the previous version and if needed, copy the contents of the previous plug-in configurations folder into the file system of the new upgraded version of Datameer.  
   INFO: Plug-in zip files from an older version are not compatible with newer versions.  Request updated plugin versions from support or your Technical Account Manager.

   # Defines the folder where to store plugin configurations
   system.property.plugin.configs.dir=<folder name>

7. If you made changes in the configuration files in your 'conf/' directory, apply the same changes to the new 'conf/' directory of your upgraded instance. 
   Example: If you made changes in 'live.properties' or 'log4j-<xyz>.properties, change them also in the upgraded instance. 

8. If needed, copy over the native libraries that you have added. 
   INFO: You don't have to copy over the native libraries that are already bundled with Datameer.

   cp -r /<old-location>/lib/native/* /<new-location>/lib/native/

9. If you made changes to the 'conductor.sh' script, apply the changes to '/<new location>/bin/conductor.sh' again. 
   Example: Changes to enable SSH values.

10. Migrate the SSL values from the previous 'start.ini' to the new one.

    # Set up a demonstration keystore and truststore
    jetty.keystore=etc/keystore 
    jetty.truststore=etc/keystore
    # Sets the demonstration passwords
    # OBF passwords aren't secure. They are only protected from casual observation.
    # See http://www.eclipse.org/jetty/documentation/current/configuring-security-secure-passwords.html
    jetty.keystore.password=OBF:1vny1zlo1x8e1vnw1vn61x8g1zlu1vn4 # storepwd
    jetty.keymanager.password=OBF:1u2u1wml1z7s1z7a1wnl1u2g # keypwd
    jetty.truststore.password=OBF:1vny1zlo1x8e1vnw1vn61x8g1zlu1vn4 # storepwd

11. If needed, copy the JVM keystore.
12. If needed, copy the JVM truststore.
13. If you use password encryption, copy the appropriate keys from the old installation folder. Find more details under 'Set Up Password Encryption in Datameer'. 
14. If needed, copy over the native libraries that you have added. 
15. Copy over the files from 'etc/custom-jars'. 
    INFO: These files could be database drivers or 3rd party libraries.
16. If needed, place Datameer X plug-ins that are not part of the standard installation package to the 'etc/custom.plugins/' directory, e.g. Advanced Governance.
17. If using custom Datameer X plugins (jar archives stored under 'etc/custom.plugins'), update those with the SDK to be in sync with the new Datameer X version.  
    
18. If existing, copy the files from the 'das-data/' folder from the old distribution to the new location.
19. Disable the 'Housekeeping/ Compaction Services'. Updating the application is finished. 

Upgrading the Database

To upgrade the database, use the script provided in the Datameer X distribution installation folder.

• For Datameer-X versions 11.0.7 and 11.1.1 or later, use the "database.sh" script by executing "./bin/database.sh upgrade" from inside the distribution installation folder.
• For earlier versions of Datameer-X 11, use the "upgrade_db.sh" script by executing "./bin/upgrade_db.sh" from inside the distribution installation folder.

The upgrade script will automatically:

• create a database dump before the upgrade (the "database.sh" script stores this in "./das-data/backups/" while the "upgrade-db" script stores it in "/bin/upgrader")
• validate the correct schema from the database
• select only necessary upgrades, SQL and Java upgrades in the correct order
• log the execution to the command line and also to a dedicated log file ("database.sh" logs to "./logs/database-tool.log" while "upgrade-db" logs to "./bin/upgrader/logs")

The Datameer X upgrade is now completed and you can launch the application.

MySQL Thread Stack Overrun During the Database Upgrade

mysqld -O thread_stack=192k

Upgrading Post-Processing

Updating YARN Path Settings

yarn class path | tr ':' ','

To check or update the path settings:

1. Start the Datameer X application and log in. The Datameer X UI opens.
    

2. Open the "Admin" tab and then "Hadoop Cluster". The 'Admin' configuration overview page opens.
    
3. Click on "Edit". The configuration page for the Hadoop cluster opens.
    
4. Navigate to the section "Cluster Settings" and then to "YARN Application Classpath" and update the 'YARN application class path' if needed.
    
5. Navigate to the section "Custom Properties" and update the 'LD_LIBRARY' path if needed.
    
6. Confirm with "Save". Updating the path settings is finished.
    

Retriggering the Search Index

Enabling SSL 

Validating the Datameer X Upgrade

Processing Workbook Validation After Upgrading Datameer

Performing the Validation

To perform the validation:

1. Start Datameer.
2. If applicable, upload recompiled plug-ins.
3. Validate the upgrade by preforming tests with baseline jobs that could previously be run without errors.
4. If applicable, test custom plug-ins.
5. Create a connection where the data is stored, import the data, add and modify the data in a workbook, and run the data.
   INFO: If no unexpected error occur the upgrade has been validated.  

Recompiling Plug-Ins

To recompile the plug-ins:

1. Check if any plug-ins are located in '<OLD Datameer X version installation folder>/etc/custom-plugins/'.
2. Request a Datameer X representative to provide these plug-ins recompiled for the Datameer X version you are upgrading to.
3. Copy the new versions of the plugins to '<NEW Datameer X version installation folder>/etc/custom-plugins/'.

Enabling Housekeeping/ Compaction Services

To enable Housekeeping/ Compaction services:

1. Perform the upgrade validation testing and make sure the upgrade is validated successful.
2. Open the file ' /<Datameer X new version installation directory>/conf/default.properties'.
3. Enable the Housekeeping Service by setting the property 'housekeeping.enabled' to 'housekeeping.enabled=true'.
4. Enable the Compaction Service by setting the property 'auto-compaction.enabled' to 'auto–compaction=true'.
5. Restart Datameer X to apply the changes.