--- title: "YARN the Apache Hadoop Resource Manager" description: "The open source framework is, therefore, free and relies on standard machines to store large amounts of data. Finally, it is possible to adapt the sys" author: "pravs bandaru" published: 2019-05-14 updated: 2019-09-07 canonical: https://www.mindstick.com/articles/65294/yarn-the-apache-hadoop-resource-manager category: "apache" tags: ["hadoop", "apache", "apache cordova", "big data", "big data analytics", "big data training", "hadoop training", "hadoop course", "hadoop online training", "hadoop certification", "hadoop training in bangalore"] reading_time: 3 minutes --- # YARN the Apache Hadoop Resource Manager **All about the Apache Hadoop [Resource Manager](https://answers.mindstick.com/qa/104857/explain-oracle-resource-manager)** ![YARN the Apache Hadoop Resource Manager](https://www.mindstick.com/mindstickarticle/95b029dc-d435-4f79-af3e-0bb4c805f71e/images/0cdbd2f2-777c-444f-a8b2-803a2b64c03d.jpeg) Within the Apache Hadoop [open source](https://www.mindstick.com/articles/337437/how-to-leverage-open-source-for-innovative-project-development) distributed processing framework, YARN is one of the main components. The purpose of this [resource management](https://answers.mindstick.com/qa/108374/what-are-the-recent-challenges-in-global-water-resource-management) and [task scheduling](https://www.mindstick.com/news/4265/chatgpt-now-supports-alarms-reminders-and-task-scheduling) technology is to allocate system resources to different applications running in a [Hadoop cluster](https://www.mindstick.com/interview/23034/hadoop-cluster-is-made-up-of-how-many-classes-of-servers). It is also used to schedule the execution of tasks on different clusters of clusters. Read More Points On **Hadoop [Online Training](https://www.mindstick.com/forum/160848/which-gadgets-are-required-for-online-training)** \ ## YARN: what is it? The initials YARN refer to the term "Yet Another Resource Negotiator", a name given with humor by the developers. This technology became a sub-project of Apache Hadoop in 2012 and was added as a key feature of Hadoop with the 2.0 update deployed in 2013. Before adding YARN, Hadoop could only run Map Reduce applications. YARN has therefore greatly increased the potential use cases of the framework. By decoupling resource management and planning from the Map Reduce [data processing](https://answers.mindstick.com/qa/114514/what-is-the-significance-of-edge-computing-in-enhancing-data-processing-and-reducing-latency) component, YARN has also enabled Hadoop to support more applications and different types of processing. For example, Hadoop clusters are now able to launch real-time analytics, streaming data, and interactive queries on [Apache Spark](https://www.mindstick.com/articles/327360/introduction-to-apache-spark) while letting Map Reduce run. Get More Points on **Hadoop Training In Bangalore** ## YARN: what are the features? YARN combines a central resource manager with containers, application coordinators, and agents to monitor the processing operations of different cluster nodes. YARN is able to dynamically allocate resources to applications based on their needs. This component of Hadoop also offers several scheduling methods: FIFO Scheduler, Fair Scheduler, or Capacity Scheduler. In addition, the [Reservation System](https://answers.mindstick.com/qa/93601/what-is-the-reservation-system-in-india) feature allows users to reserve cluster resources in advance to ensure that important processing tasks are run smoothly. Another notable feature, added with Hadoop 3.0, is YARN Federation. This increases the number of nodes that a single implementation of YARN can support by connecting different "sub-clusters" with their own resource managers. Get More Info On **Hadoop Course** YARN consists of several main components. The purpose of the Resource Manager is to accept user-submitted tasks, schedule tasks, and allocate resources to them. On each node, there is a Node Manager whose role is to monitor and report to the Resource Manager. There is also an Application Master, created for each application, to negotiate resources and work with the Node Manager to run and monitor tasks. Finally, resource containers are controlled by Node Managers and assign resources allocated to individual applications. Generally, YARN containers are organized into nodes and programmed to perform tasks only if [resources are available](https://answers.mindstick.com/qa/112555/what-resources-are-available-for-gifted-and-talented-students) to do so. Under Hadoop 3.0, however, it is possible to create "opportunistic containers" that can be placed on hold until resources are released. This concept makes it possible to optimize the use of resources. Read More Info On **Hadoop Certification** \ --- Original Source: https://www.mindstick.com/articles/65294/yarn-the-apache-hadoop-resource-manager Copyright © MindStick Software Pvt. Ltd. This Markdown version is provided for developers, AI systems, and offline reading.