we are introduced to the HBase architecture and examine the most basic
component of the architecture that is RegionServer. We have seen that using
auto-sharding the Hbase system automatically split these database table and distribute
the load to another region server when a certain configuration table limit is
reached. Now we identify how exactly Hbase system does all this mechanism on
the basis of regions.
we have Hbase Architecture model, from the previous post.
are one thing, but we also have to take a look at how individual regions work.
In HBase, a table is both spread across numerous RegionServers as well as being
made up of individual regions. As tables are split, this splits become regions.
Regions store a particular range of key-value pairs, and every RegionServer
manages a configurable number of regions. But how do an individual region look
like? HBase is a column-family-oriented data store, so how do the individual
regions store key-value pairs on the basis of column families they belong to?
The below figure (second figure) begins to answer these questions and helps us
digest more vital information about the architecture of HBase.
is written and implemented in Java — likewise the vast majority of Hadoop
technologies. Java is an object oriented programming (OOPS) language and an
ideal technology for distributed computing. So, as we continue to find out more
about HBase, remember that each of the component in the architecture is
ultimately Java object.
First off, all second figure gives us a pretty clear picture of what region objects actually look like, generally speaking. This Figure also makes it clear for us, that the regions separate data into column families and stores the data in the HDFS using HFile objects. When clients stores key-value pairs into the data tables, the keys are processed so that data is stored on the basis of column family the pair belongs to. As shown in the figure, every column family store object has two more components:
A read cache known as the BlockCache.
A write cache known as the MemStore.
The BlockCache helps us with random read
performance. Data is read in blocks from the HDFS and stored in the BlockCache.
Subsequent reads for the data — or data stored in close proximity — will be
read from RAM instead of disk, which improves overall performance. The Write
Ahead Log (WAL, for short) make sures that our HBase writes are
reliable. There is one WAL
heed the Iron Law of Distributed Computing: A failure isn’t the exception — it’s
the norm, especially when clustering hundreds or even thousands of servers.
Google considered the Iron Law in designing BigTable and HBase followed suit. But,
node failures are handled in HBase and we use WAL as a key part of this overall
we write or modify data in HBase, the data is first persisted to the WAL, which
is stored in the HDFS, and then the data is written to the MemStore cache. At
configurable intervals, key-value pairs stored in the MemStore are written to
HFiles in the HDFS and afterwards WAL entries are erased. If a failure occurs
after the initial WAL write but before the final MemStore write to disk, the
WAL can be replayed to avoid any data loss.
Figure shows two HFile objects in each column family. The design of HBase is to
flush column family data stored in the MemStore to one HFile per flush. Then at
configurable intervals HFiles are combined into larger HFiles. This strategy queues
up the critical compaction operation in HBase, which we learn in the next post.