Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

Upgrading From an Older Release

...

No Format
#Detect databases
detect old database [/Users/marko/Desktop/Datameer-2.0.1-0.20.2/das-data/database/hsql-db]
detect new database [/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data/database/hsql-db]

Upgrade datameer database version [2.0.0.6] to version [2.0.2.0]

#Assert datameer is running
No

#Copy Das Data
copy das-data from [/Users/marko/Desktop/Datameer-2.0.1-0.20.2/das-data] to [/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data]

#Upgrade starts...
Process upgrade scripts on database jdbc:hsqldb:file:das-data/database/hsql-db
	upgrade-2.0.1.hsql

#Update schema-version
set system.schemaVersion on database [/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data/database/hsql-db] to [2.0.1]


#Update data uri's [/Users/marko/Desktop/Datameer-2.0.1-0.20.2/das-data] -> [/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data]

log4j:WARN No appenders could be found for logger (org.hibernate.type.BasicTypeRegistry).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Converting 12 data entries
file:/Users/marko/Desktop/Datameer-2.0.1-0.20.2/das-data/fileuploads/1/1 -> file:/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data/fileuploads/1/1
file:/Users/marko/Desktop/Datameer-2.0.1-0.20.2/das-data/fileuploads/2/2 -> file:/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data/fileuploads/2/2
file:/Users/marko/Desktop/Datameer-2.0.1-0.20.2/das-data/fileuploads/3/3 -> file:/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data/fileuploads/3/3
file:/Users/marko/Desktop/Datameer-2.0.1-0.20.2/das-data/fileuploads/4/4 -> file:/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data/fileuploads/4/4
file:/Users/marko/Desktop/Datameer-2.0.1-0.20.2/das-data/fileuploads/5/5 -> file:/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data/fileuploads/5/5
file:/Users/marko/Desktop/Datameer-2.0.1-0.20.2/das-data/workbooks/6/6 -> file:/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data/workbooks/6/6
file:/Users/marko/Desktop/Datameer-2.0.1-0.20.2/das-data/workbooks/7/7 -> file:/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data/workbooks/7/7
file:/Users/marko/Desktop/Datameer-2.0.1-0.20.2/das-data/workbooks/8/8 -> file:/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data/workbooks/8/8
file:/Users/marko/Desktop/Datameer-2.0.1-0.20.2/das-data/workbooks/9/9 -> file:/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data/workbooks/9/9
file:/Users/marko/Desktop/Datameer-2.0.1-0.20.2/das-data/fileuploads/10/10 -> file:/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data/fileuploads/10/10
file:/Users/marko/Desktop/Datameer-2.0.1-0.20.2/das-data/workbooks/11/11 -> file:/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data/workbooks/11/11
file:/Users/marko/Desktop/Datameer-2.0.1-0.20.2/das-data/fileuploads/10/12 -> file:/Users/marko/Development/Java/datameer/dap/build/dist/Datameer-2.0.2-0.20.2/das-data/fileuploads/10/12

Check or Update YARN Path Settings

 Check or update YARN application classpath and LD_LIBRARY path.  The  path settings are used to lookup and load libraries , therefor  incorrect path settings may lead into unexpected behavior .

...

Note
iconfalse

Example:

When upgrading from CDH 4.x to CDH 5.x the classpath was renamed.
 

Code Block
$HADOOP_CONF_DIR, $HADOOP_COMMON_HOME/*, $HADOOP_COMMON_HOME/lib/*, $HADOOP_HDFS_HOME/*, $HADOOP_HDFS_HOME/lib/*, $HADOOP_MAPRED_HOME/*, $HADOOP_MAPRED_HOME/lib/*, $YARN_HOME/*, $YARN_HOME/lib/*

to

Code Block
 $HADOOP_CONF_DIR, $HADOOP_COMMON_HOME/*, $HADOOP_COMMON_HOME/lib/*, $HADOOP_HDFS_HOME/*, $HADOOP_HDFS_HOME/lib/*, $HADOOP_MAPRED_HOME/*, $HADOOP_MAPRED_HOME/lib/*, $HADOOP_YARN_HOME/*, $HADOOP_YARN_HOME/lib/*


The correct YARN classpath can be found for the Datameer client via command:

Code Block
yarn classpath | tr ':' ','

Recompile Plug-ins

Custom plug-Ins which were created by using the plug-in SDK need to be recompiled. Adjust the the versioning and  and create the plug-in againunder  under the new plug-in SDK.

All plug-ins used that aren't a part of standard distribution, e.g., Advanced Governance package, require an upgrade.

  1. Check if any plug-ins are located in <OLD Datameer version installation folder>/etc/custom-plugins/.
  2. Request a Datameer representative to provide these plug-ins recompiled for the Datameer version you are upgrading to.
  3. Copy the new versions of the plugins to <NEW Datameer version installation folder>/etc/custom-plugins/.

Validate the Upgrade

After the upgrade has completed, preform a test to validate that the upgrade was successful.

...

How to Roll Back to a Previous Database Version

When upgrading, a database dump is created by Datameer, the file name is similar to mysql_db_dump_<oldVer>.sql. If an upgrade failed (failing upgrade scripts / failing upgrade injectors during startup), the database paths might be conflicting, it is therefore necessary to restore the Datameer MySQL database dump. Follow the steps to complete this task.

  1. Drop the new database, for example on the database server or the MySQL CLI.

    No Format
    DROP DATABASE dap;

     

  2. Using the mysql-init.sql script, create a new database.

    No Format
    mysql [-h <dbhost>] -u root -p < bin/mysql-init.sql

     

  3. Import and restore the previous database using the following shell command.

    No Format
    mysql [-h <dbhost>] -u root -p dap < /path/to/dump_file_name

...