...
Scheduling Jobs with the Cluster
...
| Info | ||
|---|---|---|
| ||
Set here how jobs are scheduled for the Hadoop cluster to help optimize jobs and priorities. |
No
...
Impersonation
If not using impersonation, you can set the scheduling of jobs for specific cluster queues at either a global or per job level.
Global
...
Level
- Open the "Admin" tab.
- Select Hadoop "Cluster Configuration" from the side menu.
Add the following property in the "Custom Properties space" section.
Panel title Tez Queue Property das.job.queue.name=<cluster queue name>
Panel title MapReduce mapreduce.job.quename=<cluster queue name>
...
Job Level
- Navigate through the job wizard Job Wizard when setting up or configuring a job.
- Add the properties listed above in the "Custom Properties space" tab.
Example:
Impersonation
| Info | ||
|---|---|---|
| ||
Datameer X users that are running impersonation don't need to set any scheduling properties in Datameer. Jobs coming from Datameer X already are labeled and all configuration for the queues are made on the Hadoop cluster itself. |
Finding the Optimal Split Size/Split Count
The optimal split size and count for a Hadoop job is calculated by Hadoop from the values for max/min split size and max/min split count.
...