Scale-Out Big Data Benchmark: ElasticSearch

ElasticSearch is an open source, full text search engine that can be run on a cluster relatively easy. It's basically like an open source version of Google Search that can be deployed in an enterprise. It should be one of the poster-children of scale-out software and is one of the representatives of the so called "Big Data" technologies. Thanks to Kirth Lammens, one of the talented researchers at my lab, we have developed a benchmark that searches through all the Wikipedia content (+/- 40GB). Elasticsearch is – like many Big Data technologies – built on Java.

We are not sure why, but installing IBM's JDK caused a lot of headaches. For some reason the JVM stopped working in the middle of our tests. We got the same behavior running Apache Spark. This could be a result of our lack of experience with the IBM JDK, or the fact that the Linux LE ecosytem is still young. To cut a long story short, we ended up useing OpenJDK 8, which is part of the Ubuntu 15.04 distribution. OpenJDK is very similar to and based upon the same code as Oracle's HotSpot JDK.

We limited the systems to one socket to avoid the issues associated with garbage collection pauses and other scaling issues. There is reason why many Java benchmarks on these massive machines are using multiple JVMs.

Elastic Search

Although the POWER8 can probably perform a bit better with the IBM JDK, performance is in the same league as the best Xeons. Meanwhile as a further point of comparison we also included the score of the Xeon D from our previous article.

Database Performance: MySQL Energy and Pricing
Comments Locked

146 Comments

View All Comments

  • LemmingOverlord - Friday, November 6, 2015 - link

    Mate... Bite your tongue! Johan is THE man when it comes to Datacenter-class hardware. Obviously he doesn't get the same exposure as teh personal technology guys, but he is definitely one of the best reviewers out there (inside and outside AT).
  • joegee - Friday, November 6, 2015 - link

    He's been doing class A work since Ace's Hardware (maybe before, I found him on Ace's though.) He is a cut above the rest.
  • nismotigerwvu - Friday, November 6, 2015 - link

    Johan,

    I think you had a typo on the first sentence of the 3rd paragraph on page 1.

    "After seeing the reader interestin POWER8 in that previous article..."

    Nice read overall and if I hadn't just had my morning cup of coffee I would have missed it too.
  • Ryan Smith - Friday, November 6, 2015 - link

    Good catch. Thanks!
  • Essence_of_War - Friday, November 6, 2015 - link

    That performance per watt, it is REALLY hard to keep up with the Xeons there!
  • III-V - Friday, November 6, 2015 - link

    IBM's L1 data cache has a 3-cycle access time, and is twice as large (64KB) as Intel's, and I think I remember it accounting for something like half the power consumption of the core.
  • Essence_of_War - Friday, November 6, 2015 - link

    Whoa, neat bit of trivia!
  • JohanAnandtech - Saturday, November 7, 2015 - link

    Interesting. Got a link/doc to back that up? I have not found such detailed architectural info.
  • Henriok - Friday, November 6, 2015 - link

    Very nice to see tests of non-x86 hardware. It's interesting too se a test of the S822L when IBM just launched two even more price competitive machines, designed and built by Wistron and Tyan, as pure OpenPOWER machines: the S812LC and S822LC. These can't run AIX, and are substantially cheaper than the IBM designed machines. They might lack some features, but they would probably fit nicely in this test. And they are sporting the single chip 12 core version of the POWER8 processor (with cores disabled).
  • DanNeely - Friday, November 6, 2015 - link

    "The server is powered by two redundant high quality Emerson 1400W PSUs."

    The sticker on the PSU is only 80+ (no color). Unless the hotswap support comes with a substantial penalty (if so why); this design looks to be well behind the state of the art. With data centers often being power/hvac limited these days, using a relatively low efficiency PSU in an otherwise very high end system seems bizarre to me.

Log in

Don't have an account? Sign up now