Wednesday, February 2, 2011

HBase vs Cassandra

Adku recently decided to migrate part of it's data to a NoSQL database in order to deal with increasing load on our MySQL database. We evaluated many options including MongoDB, Amazon SimpleDB, and a few others, but ultimately narrowed the options to just Cassandra and HBase. We experimented with both databases and evaluated them more deeply and ultimately decided to use Cassandra. This post explains the high level reasons why we chose Cassandra.

Disclaimer: While these decisions apply to Adku, they might not apply to your situation. Always do your own investigation and experimentation before choosing any large part of your system.
  1. Reliability
    - We wrote a stress test tool to simulate how the databases would behave under high load. We used the default minimalistic configuration for each database following their respective documentations. Our stress test inserted 1 million rows of the full "Alice in Wonderland" text (~180kb) inserted into each row done continuously with one concurrent thread into a 3 node cluster. Surprisingly, the HBase region servers actually crashed on us consistently. Cassandra never crashed once. Although we would obviously scale the cluster under high loads and crashing nodes, this is still definitely concerning and a large win for Cassandra.

  2. Performance
    - We also wrote a simulation tool to test a more realistic scenario to see how each database would respond. Similar to the above, our tool inserted 1 million rows with 2000 characters of ascii text inserted into each row done continuously with one concurrent thread into a 3 node cluster. HBase averaged 507 microseconds per write where Cassandra averaged 480 microseconds per write. We interpreted this as basically equivalent performance so there was no real winner here.

  3. Consistency
    - Consistency is not a hard requirement for our specific use case so Cassandra's eventual consistency model is fine for us. If we do end up needing consistency, Cassandra can support it using their configurable CAP model, we would just have to take a performance hit. HBase has consistency so it is technically the winner here, but since it's not a hard requirement for us, it doesn't carry much weight.

  4. Single Point of Failure
    - Hadoop's namenode which HBase depends on is a single point of failure. This means that if the namenode goes down, the entire database is unreachable. All Cassandra nodes are identical so there is no single point of failure. This is a win for Cassandra.

  5. Hot Spot Problem
    - Our relevant row keys are currently all timestamps. HBase chooses the node to store data on by row key in sorted order. Cassandra by default stores them on a random node in random order. This means that HBase will fall into a hotspot problem where one node is handling most of the write traffic. Cassandra, however, distributes the load across all nodes evenly. This is a win for Cassandra.

  6. MapReduce
    - HBase is built on top of HDFS and Hadoop. This means that MapReduce is very easy. Cassandra supports MapReduce, but doesn't support streaming MapReduce so you have to write them in Java. I'm also not sure what the relative performance is. Cassandra supports data locality so that MapReduces tasks end up processing data on the same machine as the MapReduce task so it's possible that performance is comparable, but I haven't done adequate tests. HBase is the winner here, but so far we haven't seen any drawbacks to running MapReduce on Cassandra (besides having to write in Java).

  7. Simpler, Hackable
    - Cassandra is a simpler implementation and much easier to hack. This is the same reason why we chose Tornado instead of Apache as our web server and we've actually made quite a few modifications to the Tornado web server as a result. Bugs are also much easier to debug. HBase by comparison is much more complicated and harder to debug and hack. This is a win for Cassandra and we've even already submitted patches back into open source Cassandra.

  8. Community Support
    - As of today, there are 175 users in the #cassandra channel on HBase by comparison only has 74. Aside from IRC, it does appear that the Cassandra community is larger and more helpful than HBase. Another win for Cassandra.


  1. Glad that you found a solution that works for you.

    Your analysis of #4, "Single Point of Failure", contains some inaccuracies, though.

    You state, regarding HBase, "This means that when the master goes down, although the system can automatically recover, this recovery takes on the order of seconds during which your database is down."

    This simply is not true. The master is not involved at all in handling of normal client requests. All data in the cluster will continue to be available, for both reads and writes, in the event of a master failure. The cluster can continue operating without any master at all. The only operations impacted will be HBase internal operations, such as region splits.

  2. On the Cassandra is simpler than HBase, I have to ask, by what measure?

    Is it by this measure:
    cassandra$ find test -iname \*.java | xargs wc -l | tail -1
    7011 total
    cassandra$ cd ../hbase_trunk
    hbase_trunk ryan$ find src/test -iname \*.java | xargs wc -l | tail -1
    52090 total

    Because it seems like you might be throwing the baby out with the bathwater on #7 there.

  3. Hi Gary H, You're absolutely right. I confused the HBase master with the Hadoop namenode. I've edited the post to correct the mistake. Thanks!

  4. Hi Ryan, We called Cassandra "simpler" because it doesn't depend on Hadoop, HDFS, and ZooKeeper which means fewer moving parts.

  5. Interesting article - thanks. I am curious - what version of HBase did you use and how did you have the cluster configured?

  6. Hey Robert, We used HBase 0.20.6 and we followed the configuration instructions in these two articles: hbas

    Unfortunately, I don't think I saved the actual configuration files we ended up with, but they were extremely minimal.

  7. Any measure of performance without lots of concurrency is not an accurate benchmark of HBase or Cassandra.

  8. Hey David,

    I would agree, but it should be able to handle a single thread just as well. Setting up a test with lots of concurrency was part of our plan, but after HBase crashed on a single thread, we basically short-circuited the performance tests.

    Besides, we weren't looking to do a very deep exhaustive benchmark of the technologies as other sites have already done, we just wanted to get a feel for which would work best for us in our situation.


  9. Hi~~

    What version of Cassandra, Hbase do you test?
    and How many cluster do you use?

    thank you~~~

  10. Hey CharSyam,

    Cassandra was version 0.7.
    HBase was version 0.20.6.
    Our tests used a single cluster of three machines each.


  11. Hi Jesse,

    just to clarify the performance: 500ms per write, 1 million records: so it took approx 6 days to run the test? They seem very slow ?

  12. Hi Mogol,

    It's in microseconds, not milliseconds.

  13. On number 8, I am on both email lists cassandra and hbase and I get way more email on hbase(I am not on the irc channels though). ie. maybe it's hard to tell...I thought the hbase community was bigger until you mentioned the irc channels(maybe it's more of which community is using what for means of communication and both communities are the same size?)

  14. oh, and another good reference is how many companies search for people who know hadoop vs. how many search for cassandra(if you add java, it is essentially a tie ;) )...

  15. DataStax Enterprise (DSE) includes a Cassandra-enabled Hive/Pig MapReduce client.

    DataStax Enterprise (DSE) is a commercial distribution of Apache Cassandra and Apache Hadoop developed by DataStax. DSE provides Hadoop MapReduce capabilities using CassandraFS, an HDFS-compatible storage layer inside of Cassandra. By replacing HDFS with CassandraFS, users are able to leverage their current MapReduce jobs on Cassandra’s peer-to-peer, fault-tolerant, and scalable architecture. DataStax Enterprise is also able to support dual workloads, allowing you to use the same cluster of machines for both real-time applications and data analytics without having to move the data around between systems.

  16. I wonder how much memory they used during stress tests and performance tests. Did you checked those or have any idea about approximate usages?

  17. Hi, I dont see any read performance metrics in your test. Is there any reason for skipping them?

  18. Hi,
    nice comparison, we did something similar, but excluded HBase and Cassandra, because they "looked" complex at a time.

    at the end we evaluated Cassandra too, but did not evaluated HBase.

    Why we choose Apache Cassandra over Riak and Tokyo Cabinet

  19. Thanks for such a great comparison! When we look at HBase and Cassandra, they can look very similar. They’re both part of the NoSQL ecosystem. Although they’re capable of handling very similar workloads, there are also quite a few differences. Cassandra is designed from the ground up to handle very high, concurrent, write-intensive workloads. HBase on the other hand, has its limitations in scalability, and may require a bit more thinking to achieve the same quality of service. More at Hadoop Online Training

  20. Il Chelsea 16-17 terzo kit dispone di una combinazione di colori tradizionale in bianco e blu. Realizzato da Adidas, la nuova terza maglia del Chelsea 2016-2017 è stato rilasciato il 29 luglio ed è impostato per funzionare come Maglia nelle Coppe.maglie calcio 2017,
    maglie calcio poco prezzo, Maglia Barcelona 2017 vendita
    Maglia Deportivo a poco prezzo shop

  21. This house flag is Officially Licensed by the NFL and is a high quality item for hanging from your porch or as an interior wall banner.Hang your Minnesota Vikings Double Sided House Flag with our banner pole or fly it with our 5' or 6' flagpoles and adjustable flag brackets.
    house divided flags nhl
    nfl house flagsTennessee Titans house divided flags
    buy Cleveland Browns house divided flags

  22. Quester is a Pakistan-based questioning answering website where people can ask questions and we try our best to provide them with the best answers. Anyone can ask any legit question in English or Roman Urdu and we provide answers in the same language format.

  23. best web To make yourselves available FNAF Friv Friv Games Thank you both for sharing Holy Mass with us each day Friv Games Twizl Juegos De FNAF when we were called out late at night and any other time and for your support and encouragement in every respect. Friv 2021 Twizl Juegos Yepi Juegos Twizl Thank you for sharing with me a few of your ideas regarding the possibility.

  24. nice blog, thank you
    Danke, dafur dass Ihr jeden Tag mit uns die Hl. Messe gefeiert habt, fur all die Reparaturen im ganzen Haus Friv 10 Friv 2018 Friv 2019 fur Euere Bereitschaft Friv 10 Friv 20 Friv 2 immer zur Verfugung zu stehen Geometry Dash Twizl Twizy Twizl Danke, dass Sie Ihr Muhen um den Aufbau des Leibes Christi mit uns teilten.

  25. Senhor Deputado Cashman, agradeço-lhe a informação. Juegos Friv Jogos Friv Friv4school A todos que partilham e trabalham sob estas mesmas convicções e princípios Jeux De Friv Juegos De Roblox Zoxy Mais uma vez, obrigada ao Parlamento por comungar da visão que informa a nova política dos consumidores Juegos Kizi Juegos Yepi Twizl Zoxy assente no mercado - a visão de um mercado de consumidores informados e capacitados que procuram e usufruem, com confiança,

  26. Wir danken Ihnen
    Friv4school Gry Friv Juegos Friv dass Sie diese Hoffnung mit uns teilen und diesen Schritt auf dem Pilgerweg des Vertrauens Friv Juegos Friv Gry Friv mit uns gegangen sind. Juegos Friv Juegos De Friv Ihnen hiermit die gewünschten Informationen vermitteln zu können. Yepi Juegos Friv wir danken Ihnen für das Interesse an unseren Produkten und hoffe

  27. This article is interesting and useful. Thank you for sharing. And let me share an article about health that God willing will be very useful. Thank you :)

    Obat gatal Kudis/gudik
    Walatra Gamat Emas Kapsul
    Vitamin Untuk Kesehatan Anak
    Penyebab sering mimisan
    Cara Mengatasi Cacar Air
    Cara Menghilangkan Kantung mata
    Obat Telinga Berkerak dan Berair

  28. Generally most payday advance organizations charge around $20 to $30 per $100 of loan. To guarantee that there are no ambiguities, demand that your last payday credit sum is explained in dollars and pennies. This will enable you to realize what you have to pay, when your payday credit is expected. Payday Loans Chicago

  29. يسعي الكثير من الباحثين الي الحصول علي خدمة الترجمة البحثية من قبل خبراء في مجال الترجمة من خلال موقع مكتبتك التي يقدم العديد من الانواع المختلفة للخدمات

  30. يسعي الكثير من الباحثين الي التعامل مع موقع مكتبتك لجودة الاعمال التي يتم القيام بها علي درجة عالية من الجودة والدقة منها خدمة اعدراد اطار نظري للبحث العلمي بجودة عالية

  31. يهتم فريق عمل موقع مكتبتك بتقديم أفضل وأدق ترجمة طبية يمكن الحصول عليها وذلك من خلال المقارنة مع المواقع الالكترونية الاخري لجذب انتباه الباحث العلمي من خلال عرض ترجمة نصوص علمية في كافة المجالات العلمية والطبية

  32. I am really happy to read this blog post, you have many information put it on this article, am really suggest to my all dear friend to visit this blog post and collect this useful and valuable information, keep sharing like this type of blog post, thank you so much for read my comment, if any one searching top 5 website designing company in India please come on my website we are doing for you any type of website design for your, business grow up your work grow up your money.
    Top 5 Website Designing Company in India

  33. Am visit of many blog but not get valuable information, but after seen your blog provide it me a lot of valuable and useful information I really like to thank you so much for share this information with us, if are inserted in grow up your life style and also grow up your looking so please come on my website am put it on many information so and visit again time thank you so much for read my comment I hope your also like my comment and you also go on my web page.
    Lifestyle Magazine India

  34. That was such an awesome content to read and going through it.Thanks for such a good information.our product related for servo voltage stabilizer and transformer.
    Servo Stabilizer Manufacturer in india

    What is Servo Voltage Stabilizer?

    Distribution Transformers Manufacturer in india

    What is Step Down Transformer?

  35. Given article is very helpful and very useful for my admin, and pardon me permission to share articles here hopefully helped :

    Cara Mengatasi Perih Di Ulu Hati Secara Alami
    Cara Mengobati Sirosis Hati Secara Alami

  36. May be useful for all, helpful article once and pardon me permission to share also here :

    Pengobatan Maag Kronis Secara Tradisional


  38. Thanks for sharing your wealthy information. This is one of the excellent posts which I have seen. I go through your all of your blog, but this blog is the best one.
    Top 5 Web Designing Company In India | SEO Services In India

  39. See the latest customer reviews, feedbacks, and complaints about our country top packers and movers. Like Agarwal Packers and Movers. How was the shifting service of Agarwal Packers and Movers? See now Agarwal Packers and Movers Customer Reviews.


  40. Wow, amazing blog layout! How long have you been blogging for? you make blogging look easy. The overall look of your website is fantastic, let alone the content!

    3d animation Company
    Best Chatbot Development Company
    Mobile app development in Coimbatore

  41. Thank you so much for share this type of articles keeps the users interest in this article, I really like to read this awesome blog.
    Transformer Manufacturers In Pune | Transformer Manufacturers In Mumbai | Transformer Manufacturers In India

  42. I am very interested in this post, everyone should know it more, wish you success. The author of this post is admirable, it brings a lot of emotion to everyone reading it, let's share it more widely.
    free friv Games 2019, Jogos de Friv, friv for school free Games, cá koi mini

  43. A number of cranes of various sizes are used to transport furniture properly.
    It has many cars that are equipped to do the job to the fullest.
    Furniture is well packaged before moving from one place to another.
    When the furniture is delivered to the desired location, the staff will install the furniture with each piece in place.شركة نقل عفش
    شركة نقل اثاث من الرياض الى جدة

    شركة نقل عفش من الرياض الى جدة

  44. Thanks for taking the time to discuss and share with us. I wish more people would read this. We have expert developers with over 12+ years of experience. If you have any requirement regarding Android & iOS mobile app development then you can contact us at +1-877-659-9068.

  45. Nice post. I learn something totally new and challenging on websites I stumbleupon everyday. It's always helpful to read articles from other authors and use a little something from their sites.
    kbc head office number

  46. Happy reading, Waiting for More and new Updates
    I Have Read Your Last Post. Thanks.

  47. I feel happy about and learning more about this topic. keep sharing your information regularly for my future reference. This content creates new hope and inspiration within me. Thanks for sharing an article like this. the information which you have provided is better than another blog.
    IELTS Coaching in Delhi

  48. Usually I never comment on blogs but your article is so convincing that I never stop myself to say something about it. You’re doing a great job Man,Keep it up.
    overseas education consultantsin delhi
    ielts coaching in gurgaon

  49. Speaker or Volume button not working.
    eCommerce Web Development Singapore
    I really enjoyed your blog Thanks for sharing such an informative post.

  50. Great information, thanks for sharing with us. Get the amazing website designing services by OGEN Infosystem in Delhi, India.
    Website Designing Company in Delhi

  51. Thanks for sharing such a great blog... I am impressed with you taking time to post a nice info.
    iPad App Development Company
    Hybrid App Development Services
    Web Development Services

  52. very nice graphics, your creativity is very impressive. it is very good information for make Creativity
    PHP Development Company
    Laravel Development Company
    CodeIgniter Development Company

  53. it has good information which i found everywhere but when i saw your article i really like it,could you please give us these type of informative blogs.
    Website Designing Company in Delhi
    Responsive Design Services
    UX Design Services

  54. I am always eager to catch hold of the new posts being published on your website, because of this i use to updated, thanks for sharing this wonderful article.
    Graphic Design Services
    Digital Marketing Services
    Best Seo Company in Delhi

  55. Alot of blogs I see these days don't really provide anything that I'm interested in, but I'm most definitely interested in this one.
    EMail Marketing Services India

  56. thanks for inspired me!agen Bola Online Terpercaya
    I really enjoyed your blog Thanks for sharing such an informative post.
    which is the knowledge that we blender reviews


  57. If you are a student of engineering dissertation help then you must need to complete your dissertation task. Our writing service gives you help to finish your work effectively, we work 24*7 for your convenience.

  58. Very interesting blog. Alot of blogs I see these days don't really provide anything that I'm interested in, but I'm most definately interested in this one. Just thought that I would post and let you know

    Agen Bola Sbobet

  59. Les punaises puantes recherchent des sites d'hivernage à la fin de l'automne pour se mettre à l'abri de l'hiver. Au printemps, ils se rassemblent souvent sur les côtés des bâtiments.
    شركة مكافحة حشرات
    شركة مكافحة النمل الابيض بالرياض
    شركة مكافحة الصراصير بالرياض

  60. which is the best assignment help services provider agency in the world. Here we provide the professional and high Experienced Writers who are proficient in writing reviews report services. Our assignment experts Writers ensure you that students can get completed assignment within the given time.

  61. Thank you for sharing this. The blog is very informative. The given blog tends to provide essential information about the changes and modifications of the Android app development company usa.

  62. . Even the topmost enterprises also engaged students in different sectors of their firm who have done this certificate training course. cursos de ti

  63. I just found this blog and have high hopes for it to continue. Keep up the great work, its hard to find good ones. I have added to my favorites. Thank You Wanna Party

  64. The message will not be the same for each audience. Audience will be grouped according to their areas of specialization and participation.freelance writer for hire

  65. I am glad after coming on this site! It has the information that I was searching from so long. Students struggling to write their academic assignments can pick for our assignment help service and can get a well written work from us.

  66. Thanks for sharing such a great blog.
    Nunc systems is a one stop destination for all kinds of Mobile Applications, we have more than 10 years of experience in the Mobile Application Development industry.
    Based out of Hyderabad we provide Android and iOS APP Design and Development services around the globe.
    Top Mobile App development Company in Hyderabad

  67. Get complete programming homework without even writing a single word using Programming help . You will get a chance to connect with an experienced programmer using Programming assignment help online services.
    Java Programming Help
    Java assignment help