Showing posts with label jobs. Show all posts
Showing posts with label jobs. Show all posts

Wednesday, 12 July 2017

How to Change the priority of Hadoop Jobs (Hive,MapReduce,pig)?

Hadoop provides a command line utility for changing the priority of running job. There are total 5 different priority level are there. These priority levels are listed below:

VERY_HIGH
HIGH
NORMAL
LOW
VERY_LOW

We can change the priority of the running hadoop job syntax is shown below:

 hadoop job -set-priority <job-id> <priority>  

Example:

 hadoop job -set-priority job_20170111540_64444 VERY_HIGH  

From the names of the priority levels, it is quite obvious that highest priority is given to the jobs whose priority level is VERY_HIGH and least priority is given to the jobs whose priority level is VERY_LOW.

Priority in Map Reduce Program:

In order to set priority for a job on Hadoop 1 cluster, you can use the following example:

 -> SET mapred.job.priority=<priority_value>;  

In order to set priority for a job on Hadoop 2 cluster, you can use the following example:

 -> SET mapreduce.job.priority=<priority_value>;  

Priority in Pig Program:

The below property is used to set the job priority is Pig Programming.

 grunt> SET job.priority 'high'  

The following <priority_value> values are supported:
very_low, low, normal, high, very_high

Priority for Hive Query:

In order to set priority for a HIVE query on Hadoop 1 cluster, you can use the following example:

 hive> SET mapred.job.priority=VERY_HIGH;  

In order to set priority for a HIVE query on Hadoop 2 cluster, you can use the following example:

 hive> SET mapreduce.job.priority=VERY_HIGH;  


If you are facing any problems in changing the priority of the hadoop jobs, then please comment here.