Enabling and Reading JDBC Debug Logs

Owned by Juliane Wetzel
Nov 12, 2019

Users have the ability to analyze issues or problems with Datameer's JDBC-based:

  • Connections
  • Import jobs
  • Export jobs

This can be accomplished by turning on debug logging for the artifacts.

Debug Logging for All JDBC Artifacts

  1. Add to the following lines to the log4j.properties file conf/log4j-production.properties.

    log4j.category.datameer.dap.common.db=DEBUG
log4j.category.datameer.dap.common.job.dapimport.jdbc=DEBUG

  2. Reboot the Datameer X server.

Example log printout:

DEBUG [2016-12-14 20:18:26.656] [JobExecutionPlanRunner] (JdbcConnector.java:956) - Database : Netezza NPS Release 7.1.0.0 [Build 35553]
INFO [2016-12-14 20:18:26.658] [JobExecutionPlanRunner] (JdbcConnector.java:157) - connected to 'jdbc:netezza://netezza.datameer.lan:5480/services' with schema set to 'null'
DEBUG [2016-12-14 20:18:26.658] [JobExecutionPlanRunner] (JdbcConnector.java:562) - Checking if table Table{_catalog=ALIFFI, _name=datameer_exp_tmp_4, _query=null, _condition=( 1 = 1 ), _fields=[], _splitSourceColumn=null} exists.
...
DEBUG [2016-12-14 20:18:37.622] [JobExecutionPlanRunner] (BatchJdbcInserter.java:82) - Preparing batch with query: INSERT INTO "ALIFFI".datameer_exp_tmp_4 (dakfl,col1,col2,col3,col4,co5,co6,co7,ID) VALUES (?,?,?,?,?,?,?,?,NEXT VALUE FOR "ALIFFI".datameer_exp_tmp_4_seq)
DEBUG [2016-12-14 20:18:37.714] [JobExecutionPlanRunner] (BatchJdbcInserter.java:43) - Inserting batch of 5000 rows.
DEBUG [2016-12-14 20:18:39.443] [JobExecutionPlanRunner] (BatchJdbcInserter.java:46) - Batch execution finished.
DEBUG [2016-12-14 20:18:39.443] [JobExecutionPlanRunner] (BatchJdbcInserter.java:82) - Preparing batch with query: INSERT INTO "ALIFFI".datameer_exp_tmp_4 (dakfl,col1,col2,col3,col4,co5,co6,co7,ID) VALUES (?,?,?,?,?,?,?,?,NEXT VALUE FOR "ALIFFI".datameer_exp_tmp_4_seq)
DEBUG [2016-12-14 20:18:39.559] [JobExecutionPlanRunner] (BatchJdbcInserter.java:43) - Inserting batch of 5000 rows.
DEBUG [2016-12-14 20:18:40.582] [JobExecutionPlanRunner] (BatchJdbcInserter.java:46) - Batch execution finished.
DEBUG [2016-12-14 20:18:40.583] [JobExecutionPlanRunner] (BatchJdbcInserter.java:82) - Preparing batch with query: INSERT INTO "ALIFFI".datameer_exp_tmp_4 (dakfl,col1,col2,col3,col4,co5,co6,co7,ID) VALUES (?,?,?,?,?,?,?,?,NEXT VALUE FOR "ALIFFI".datameer_exp_tmp_4_seq)

These log entries can be found in the job.log of the job-trace-log when Datameer X is configured in the local execution mode. They can be found in the YARN application logs when Datameer X is configured to execute against Tez, Spark, or MapReduce.

Debug Logging for Specific Jobs

You can also configure debug logging on the job level to target specific artifacts. A reboot of Datameer X isn't required.

  1. Create a new artifact or edit an existing artifact.
  2. On the Schedule page, expand the Advanced section.
  3. From the Default log severity menu, select Debug.

  4. In the Logging Customization field, enter the following properties:

    log4j.category.datameer.dap.common.db=DEBUG
log4j.category.datameer.dap.common.job.dapimport.jdbc=DEBUG
  5. Save the artifact.