LATEST BLOG TAGGED ON "HADOOP"

Data Science Analytics Certification for Data Governance

Data governance becomes very important in such scenarios. What’s data governance? It’s the complete administration of usability, integrity, availability, and data security in an organization.

HDFS vs. HBase : All you need to know

It has created the need for a more organized file system for storage and processing of data, when it is observed a sudden increase in the volume of data from the order of gigabytes to zettabytes.

Hadoop 

Switching from Java to Big Data/Hadoop Career: The Whys and Hows

Once in a while, there may come a feeling that you are stuck in the same job profile and living a monotonous professional life. This generally leads to the realization that a change in your profile is much needed.

HBase Architecture: Introduction and RegionServers(Part-1)

The reason that folks such as chief financial officers are excited by the thought of using Hadoop is that it lets us store massive amounts of data across a cluster of low cost commodity servers — that’s music to the ears of financially minded people.

Big Data: HBase as Distributed, Persistent, Multidimensional Sorted Map

Now we are very well familiar with the power packed characteristics and nature of Hbase.

Pig Script Interfaces and Mode of Running in Hadoop

All scripts are run on a single machine without requiring Hadoop MapReduce and HDFS. This can be useful for developing and testing Pig logic.

Introduction to Ooize in Hadoop

Moving data and running different kinds of applications in Hadoop is great stuff, but it’s only half the battle.

Clustering and Classification with Mahout

Unlike the supervised learning method described earlier for Mahout’s recommendation engine feature, clustering is a kind of unsupervised learning — where the data labels points are not known ahead of time and should be inferred from the data without

Statistical Analysis in Hadoop

Big data is all about applying analytics to more data, for more people.

MapReduce Driver Class:

Although the mapper and reducer implementations are all we need to perform the MapReduce job, there is one more piece of code necessary in MapReduce:

Pig Data Types in Hadoop

We have already seen the Pig architecture and Pig Latin Application flow. We also learn the Pig Design principle in the previous post.

Pig Design Principles in Hadoop

Pig Latin is the programing platform which provides a language for Pig programs. Pig helps to convert the Pig Latin script into MapReduce tasks that can be run within Hadoop cluster.

Introduction to Pig in Hadoop

Java MapReduce programs and the Hadoop Distributed File System (HDFS) provide us with a powerful distributed computing framework, but they come with one major drawback

YARN’s Resource Management

The most key component of YARN is the Resource Manager, which governs and maintains all the data processing resources in the Hadoop cluster.

Hadoop File System Commands: ls Command Output Analysis

In my previous post, I have explained various Hadoop file system commands, in which I also explained about the “ls command”.

HDFS Architecture in Hadoop

The core concept of HDFS is that it can be made up of dozens, hundreds, or even thousands of individual computers, where the system’s files are stored in directly attached disk drives.

Data Replication in Hadoop: Slave node disk failures (Part -2)

Hadoop was originally designed with an intention to store petabyte data at the scale, with any Potential limitations to scaling out are minimized.

Three modes of Hadoop Cluster Architecture

Hadoop is primarily structured and designed to be deployed on a massive cluster of networked systems or nodes,

Why we need Map-Reduce in Hadoop?

After we have stored piles and piles of data in HDFS (a distributed storage system spread over an expandable cluster of individual slave nodes),

Writing and Reading Data from HDFS

For creating new files in HDFS, a set of process would have to take place (refer to adjoining figure to see the components involved

HDFS Federation and High availability

Before Hadoop 2 comes to the picture, Hadoop clusters were living with the fact that Name Node has placed limits on the degree to which they could scale.

Various Data compression codecs in Hadoop

Here we enlist and identify some common codecs that are supported by the Hadoop framework.

Storing Data in HDFS

Just to be clear, storing data in HDFS is not entirely the same as saving files on your personal computer.

Hadoop Toolbox

Besides, the major contribution of Amazon EMR services and its other related tools, many other companies also provide certain useful Hadoop Tools enlisted as following:

Concept of Map Reduce in Hadoop

Though MapReduce as a technology is relatively new, it builds upon much of the fundamental work from both mathematics and computer science,

NEWSLETTER

Enter your email address here always to be updated. We promise not to spam!