Smart Execution Administration Tools

Datameer X administrators have the ability to configure Smart Execution^TM properties.

Troubleshooting

Force framework execution

Datameer's Smart Execution™ dynamically chooses the best execution engine for each of your specific jobs. But, you have the option to force a specific framework if you are experiencing a problem with performance or believe that another framework would be better suited for one of your jobs. 

The property that selects which execution framework is das.execution-framework=<framework>. The default value for this property is Smart which allows Datameer's Smart Execution feature to select the framework for each task. This property value can be changed to force a specific framework.

•  Force a job to be run with the Optimized MapReduce (Tez) execution framework by adding the following property to the Hadoop properties on the cluster configuration page: 

  das.execution-framework=Tez

Performance Tweaking

Vcores per node

Select the number of cores on each node that are used to process a job.

Example: If you enter a value of 10 and also have 10 nodes, 100 task containers could be used to process the job depending on queue availability.

das.yarn.available-node-vcores=<number of vcores per node>

Session pooling 

Session pooling reuses Tez sessions across Datameer X jobs and can help elevate processing strain as well as helping to avoid container start-up cost. 

Enable session pooling with the following property:

das.tez.session-pool.max-cached-sessions=<number of nodes>

The parameter das.tez.session-pool.max-cached-sessions controls the maximum number of idle sessions held in cache. If it is set to zero, there is no session pooling. If it is set to auto, it uses the number of nodes in the cluster as value for the maximum number of sessions.

Ideally the Tez sessions are not limited to the number set in the property, so it is elastic. The number of Tez sessions increases as necessary, but excess sessions are closed once the processes are complete.

The property das.tez.session-pool.max-cached-sessions is set to 25 by default, but that doesn't mean it is restricted to creating only 25 Tez sessions. If required, more Tez sessions are spawned. The property means it only cache 25 Tez sessions in total, so if the cluster size is good enough then you can also increase the cached sessions. If there are no cached sessions, then a new session is spawned.

You can go to N number of Tez sessions (if the cluster and Datameer X node have resources to do so) but, Datameer X keeps only the configured sessions. These sessions run even if there are no jobs submitted by Datameer. It is similar to a cache for memory. If the cache gets hit then it is good, if there is cache miss then it spawns a new cache as necessary.

Depending on the usage of your system and to double-check your assumption you can set the property das.tez.session-pool.max-cached-sessions=0. This setting means no sessions run if there are no jobs submitted by Datameer. If a job gets submitted to the cluster, a session is spawned, which has a small latency to bring up the application JVM for the job and tasks only.

The session pool acts similar to cache. To reduce the latency from bringing up containers for small applications, the maxIdleTime is set to two minutes and the maxTimeToLive for a session is set to two hours. These default values are well set for most small applications and Datameer X does not recommend to change them.

It is possible to change the properties if needed:

• das.tez.session-pool.max-idle-time=2m 
• das.tez.session-pool.max-time-to-live=2h 

These are found in <datameer-install-path>/conf/default.properties. 

All sessions maintained by the session pool have the container reuse enabled. Timeouts for the containers are not set. The Tez default setting being used is 10s.