Wednesday, 12 July 2017

How to Change the priority of Hadoop Jobs (Hive,MapReduce,pig)?

Hadoop provides a command line utility for changing the priority of running job. There are total 5 different priority level are there. These priority levels are listed below:

VERY_HIGH
HIGH
NORMAL
LOW
VERY_LOW

We can change the priority of the running hadoop job syntax is shown below:

 hadoop job -set-priority <job-id> <priority>  

Example:

 hadoop job -set-priority job_20170111540_64444 VERY_HIGH  

From the names of the priority levels, it is quite obvious that highest priority is given to the jobs whose priority level is VERY_HIGH and least priority is given to the jobs whose priority level is VERY_LOW.

Priority in Map Reduce Program:

In order to set priority for a job on Hadoop 1 cluster, you can use the following example:

 -> SET mapred.job.priority=<priority_value>;  

In order to set priority for a job on Hadoop 2 cluster, you can use the following example:

 -> SET mapreduce.job.priority=<priority_value>;  

Priority in Pig Program:

The below property is used to set the job priority is Pig Programming.

 grunt> SET job.priority 'high'  

The following <priority_value> values are supported:
very_low, low, normal, high, very_high

Priority for Hive Query:

In order to set priority for a HIVE query on Hadoop 1 cluster, you can use the following example:

 hive> SET mapred.job.priority=VERY_HIGH;  

In order to set priority for a HIVE query on Hadoop 2 cluster, you can use the following example:

 hive> SET mapreduce.job.priority=VERY_HIGH;  


If you are facing any problems in changing the priority of the hadoop jobs, then please comment here.

19 comments:

  1. Really Good blog post.provided a helpful information.I hope that you will post more updates like this Big Data Hadoop Online Training Bangalore

    ReplyDelete
    Replies
    1. Hadoop provides a command line utility for changing the priority of running job. There are total 5 different priority level are there. These priority levels are listed below:

      VERY_HIGH
      HIGH
      NORMAL
      LOW
      VERY_LOW

      Big Data Projects For Final Year Students

      Cloud Computing Projects Final Year Projects



      python projects for final year students

      Delete
  2. Pleasant Tips..Thanks for Sharing….We keep up hands on approach at work and in the workplace, keeping our business pragmatic, which recommends we can help you with your tree clearing and pruning in an invaluable and fit way.
    Digital Marketing Training in Chennai

    Digital Marketing Training in Bangalore

    digital marketing training in tambaram

    digital marketing training in annanagar

    ReplyDelete
  3. Thanks for splitting your comprehension with us. It’s really useful to me & I hope it helps the people who in need of this vital information. 

    Data Science Training in Chennai | Data Science course in anna nagar
    Data Science course in chennai | Data science course in Bangalore
    Data Science course in marathahalli | Data Science course in btm

    ReplyDelete
  4. The site was so nice, I found out about a lot of great things. I like the way you make your blog posts. Keep up the good work and may you gain success in the long run.
    Python training in marathahalli | Python training institute in pune

    ReplyDelete
  5. I read this post two times, I like it so much, please try to keep posting & Let me introduce other material that may be good for our community.

    java training in chennai | java training in bangalore

    java online training | java training in pune

    ReplyDelete
  6. This comment has been removed by the author.

    ReplyDelete
  7. This comment has been removed by the author.

    ReplyDelete
  8. Thank you so much for this nice information. Hope so many people will get aware of this and useful as well. And please keep update like this.

    Big Data Solutions

    Data Lake Companies

    Advanced Analytics Solutions

    Full Stack Development Company

    ReplyDelete
  9. Data migration services offered by your company have helped in passing all the series of functions entirely, which are necessary for data transformation.

    ReplyDelete
  10. AWS big data consultant should understand the need of Data, and they should work to build more appropriate services to meet the requirements of their clients.

    ReplyDelete

  11. I was just examining through the web looking for certain information and ran over your blog.It shows how well you understand this subject. Bookmarked this page, will return for extra. data science course in vadodara

    ReplyDelete