HBase is by design a columnar store, that is optimized for random reads.
\nYou just ask for a row using rowId as an identifier and you get your
\ndata instantaneously.<\/p>\n
Performing a scan on part or whole table is a completely different thing.
\nFirst of all, it is sequential. Meaning it is rather slow, because it
\ndoesn\u2019t use all the RegionServers at the same time. It is implemented
\nthat way to realize the contract of Scan command \u2013 which has to return
\nresults sorted by key.<\/p>\n
So, how to do this efficiently?<\/p>\n
<\/p>\n
The usual way of getting data from HBase is with the help of its API,
\nmainly Scan objects. To accomplish the task I\u2019ll use just them. I\u2019ll
\nspecify startRow and stopRow, so that each Scan request will be looking
\nthrough only part of the key space.<\/p>\n
It is crucial to note, that this whole solution works because of key
\nsorting properties in HBase. So, HBase scans a table according to ascending key
\nvalues. Since keys are of String type, key with value \u201c1\u201d is smaller
\nthan \u201c2\u201d, because they are sorted lexicographicly. So, also key with value \u201c12345\u201d is smaller than \u201c2\u201d. I\u2019ve
\nleveraged this property so that I\u2019ve partitioned my whole key space according to
\nthe first character of the key. In my case keys contain only digits. So I
\nhave 10 ranges:<\/p>\n
null-1\r\n1-2\r\n2-3\r\n3-4\r\n4-5\r\n5-6\r\n6-7\r\n7-8\r\n8-9\r\n9-null<\/pre>\nThe speedup comes from the fact, that each range resides in its own
\npartition. That\u2019s right, I\u2019ve presplit the table to have 10 partitions.
\nThis corresponds rather nicely with my cluster\u2019s setup, because I have
\nmore than 10 RegionServers. So every partition should be on different
\nRegionServer. It will allow the code to do the requested scan operations
\nin parallel \u2013 giving me this exact performance boost.<\/p>\nHow I\u2019ve created the input table:<\/p>\n
$ create 'tariff_changes', { NAME => 'cf', SPLITS_FILE => 'splits.txt', VERSIONS => 50, MAX_FILESIZE => 1073741824 }\r\n\r\n$ alter 'tariff_changes', { NAME => 'cf', SPLITS_FILE => 'splits.txt', VERSIONS => 50, MAX_FILESIZE => 1073741824 }<\/pre>\nSplit file is just something along this lines:<\/p>\n
1\r\n2\r\n3\r\n4\r\n5\r\n6\r\n7\r\n8\r\n9\r\n0\r\n<\/pre>\nThis tells HBase what are the rowKeys starting and ending each of the
\npartitions.<\/p>\nOk, so after this rather lengthy introduction, what the actual code
\ndoes? It just spins of a few threads \u2013 one for each partition \u2013 and runs
\na Scan request tailored to that partitions key space. This way, I got a
\n10x speedup for this particular scan. The execution time went down from
\n30 minutes to 3 for my use case.<\/p>\n