The very first independent Nested Paging Virtualization tests
by Johan De Gelas on July 10, 2008 12:00 AM EST- Posted in
- IT Computing general
The first generation of hardware virtualization was nothing to write home about from a performance point of view. In many cases, hardware virtualization was disabled by the OEMS to avoid problems, in other cases hardware virtualization was simply slower than the current forms of software virtualization. It was a step back from the highly optimized VMWare binary translation, but of course a step back to take many steps forward.
We explained this in great detail here. One of the problems was that software virtualization tackled (virtual) memory management so much better than hardware virtualization. First generation Hardware virtualization - as can be found in the Xeon 53xx series and second generation Opterons - should be seen as way of making it easier to write a slim hypervisor. One of the examples is Linux KVM, which makes good use of hardware virtualization while it avoids the complexities of binary translation and paravirtualization. Basically many of the new hypervisor solutions would not be even possible without the availabiltiy of hardware virtualization capable CPUs.
The second generation of hardware accelerated virtualization which was first introduced in the third generation of Opterons (AMD "Barcelona" aka K10) is a lot better from a performance point of view. It offers massive tagged TLBs that can support the memory management of the different guest OS and the hypervisor. Intel's Nehalem will use this technique too.
The past months, we have been working very hard on a massive virtualization benchmarking effort. It will take a while to analyze the huge amount of data we generated so far, but we could not resist to give you already a taste.
The benchmarks below have been done on a dual socket Opteron 8356 (so 8 cores at 2.3 GHz) machine. Our quad socket board is not supported by ESX, so that is why we limited our Opteron 8356 to 2 CPUs for now. The specific benchmarks below were done on Xen 3.2.0 which is distributed with SLES 10 SP2. Four virtual machines are running at the same time, and each virtual machine has 2 virtual CPUs which are firmly linked to 2 physical Cores. Two webservers, one Oracle OLTP test and one DSS MySQL test are all running in parallel on our dual opteron server. Each virtual machine has 4 GB of RAM and runs Windows 2003 R2.We will give a lot of more benchmarking detail in the upcoming article.
We enabled and disabled HAP in the kernel parameters of Xen 3.2.0 (SUSE SLES 10 SP2). Oh yes, Hardware Assisted Paging is the same thing as Nested Paging, RVI (AMD marketing) and EPT (Intel marketing). Confused? Blame Intel and AMD marketing :-). The No-HAP or No NPT benchmarks are normalized to 100%.
There are two very interesting analyses to be made from the graph above. First of all, Xen 3.2.0 which can be found in the newest Novell SLES 10 SP2, is capable of running Windows 2003 R2 under heavy stress.
In theory this has been the case for quite some time as it was claimed from the start of Xen 3.0 that Xen would be able to run Windows based virtual machines thanks to hardware virtualization. In practice, all Xen versions 3.0.x have a lot of trouble with Windows as a guest OS, freezing the virtual machines up etc. Simply put, running Windows on a Xen hypervisor before Xen 3.2 is simply asking for trouble.
But now Xen is capable of running Windows 2003 machines. Our in house developed software really stresses the virtual machines well over their maximum, so we have good reason to believe that if a software survives the stresstest hell we put it through, it is stable. We believe Xen is now a viable (and probably cheap) alternative to VMWare ESX and Hyper-V.
Secondly, Nested Page Tables, also know as Rapid Virtualization Indexing (Yep, that is the marketing name :-) can give a serious boost to some memory intensive webservers. 31% better performance is nothing to sneeze at. Let me remind you that these webservers are not some industry standard benchmarks for which AMD can sent over a team of engineers to optimize it.
Now, we just used a realworld webserver for this benchmarking. If you like to get an idea of what kind of webserver software we tested, look here.
The OLTP test ("Oracle Swingbench") is not limited by the harddisk performance. We test with six 15000 rpm seagate SAS 300 GB disks (more or less the fastest harddisk you can get) in RAID-0 and we have performed the same test with twelve disks. The extra disks gave a 1% boost, so our disk system is not the bottleneck.
The Oracle Swingbench OLTP tests shows a smaller performance boost thanks to NPT, 7%. Not bad at all.
The two benchmarks shows that the second generation of hardware virtualization does make a difference. So hardware virtualization is one of those new technologies that really has a promising future. Seeing how NPT really pushes hardware virtualization forward feels a bit similar to the way the Voodoo 2 opened a new world of 3D graphics, or how the first dualcore CPUs started the multicore race. Pretty exciting if you ask us, and we feel the same way about our upcoming articles.
Oh and in case you haven't noticed, it.anandtech.com really offers you some very special content on virtualization. If you are sick an tired of reading the same woolly - copied from the software vendors - texts about virtualization, you should start reading right here. We have
explained for you. Read those 3 articles and you are up to speed on what is going in the virtualization market. Next some very unique benchmarks on Hyper-V, VMWare ESX and Novell/Xen. AMD versus Intel will be - of course - in there too...
Facebook
Twitter
RSS
20 Comments
View All Comments
computerfarmer - Saturday, July 12, 2008 - link
This is my first read into this area. I also read the three articles posted at the bottom of this one. Information is the key to understanding and these articles help with this. Like any assignment at university, it will lead you to further studies for a better understanding under different conditions.This article and the comments, have given me more insight into this area.
Thank you all
sht - Friday, July 11, 2008 - link
You mention KVM yourself Johan, and I'd be very interested if you had a look on it. Any reasons not to include it in the upcomming article? Any timeline on when we might see benchmarks on virtualization including it?Thanks.
iwodo - Thursday, July 10, 2008 - link
While I understand most current use for Virtualization is in Servers. Would they one day come to gaming / graphics. Being able to use WindowsXP inside Virtual Machine with FULL Graphics power.i.e GPU Virtualization. Possible?
BaronMatrix - Friday, July 11, 2008 - link
That is coming with IOMMU by AMD (the Intel version escapes me). It will probably NOT enable DX10, but it will allow better memory management and throughput for graphics. SO you won't have an emulated S3 graphics card.yyrkoon - Thursday, July 10, 2008 - link
You may want to consider fixing the RSS Feed link to container based OS virtualization, as it has not worked for me since day 1. The link from here however does work.I would also like to add that Xen *IS* free, not cheap, as you can download a binary, or the source from xen.org HVM(hardware virtual machine) have also been possible for over a year, and is supported on all core 2 duo up CPUs, and AMD AM2 on up CPUs. You guys should really check our irc.freenode.org ##xen(if you have not already) and speak to the vetrans who have been running xen para vitual, or hardware virtual for years now.
*shakes head at running MySQL/php on a Win2k3 box*
rippleyalienshakd - Thursday, July 10, 2008 - link
In all fairness, this test is not even close to being realistic.1. How many web servers does an organization run?
2. How many SQL servers does an organization run?
3. XEN? Yah XEN is free, the free version.
I Have to tip my hat to VMware. YAH it costs..
BUTTTTT there are things i can do on a vmware server that XEN, rather the freebe, Citrix XEN, even SUN XEN.. They are close, but not even within the same league.
I actually have VMware VIRTUAL servers that out- preform Physical servers.
NEXT time you do something like this, how about give more details. The test that you preformed, is amateur at best.
there are 3 main reasons to even think about virtualization.
1. Legacy server consilidation
2. Disaster scenerios
3. Virtual Infrastructure
Seeing how 4 web servers preform on 1 physical server doesn't make sense. Even Microsoft put out a benchmark on Exchange 2007 on VMware ESX. 10,000 users, on 2 VMware ESX servers. now that my friend is a benchmark.
NEXT demo/test, use the actual parts. You have failed to show actual cpu/memmory utilization. that is a XEN drawback. CPU outstanding, but Memmory utilization horrible.
The ability to do accurate CPU sharing (Resource sharing) is HORRIBLE. Network configurations and throughput, HORRIBLE
JohanAnandtech - Friday, July 11, 2008 - link
It is a blogpost, not a full article with full benchmark configuration disclosure. That is mentioned more than once in this post.Tests were performed based on the input of real companies. Yes, in some cases, just putting a lot of mailservers together might be more relevant. But as you indicate yourself, microsoft and VMWare produce more than enough exchange benchmarks. So if we can chose, we chose a benchmark scenario that is realistic, but not beaten to dead.
Virtual servers can only outperform physical servers if your application scales badly over more cores. You load balance your load over several virtual machines, and compare this with with one application that has to scale over many CPUs. But otherwise virtual servers can not outperform physical ones.
I am not saying Xen is better than VMWare. I am just saying we found out that Xen is finally ready to run Windows. I have seen a lot of claims that it should work on Xen 3.0.x, but we found that it was impossible to run any kind of Windows on that version. But with Xen 3.2, it seems to be very stable.
aenagy - Monday, July 14, 2008 - link
We run on ESX Server 3.0.2 and I have been resisting attempts to move some of the heavier applications such as SQL Server 2005, Exchange Server 2003 and other LoB applications due to concerns about performance. I am hoping that the combination of upgrading ESX Server 3.5 plus NPT plus IOMMU would provide the necessary throughput for these applications.So please include the latest version of ESX Server in your benchmarks. Also, even though the vendors have thier own whitepapers on SQL Server and Exchange Server I would still like to see some independent benchmarks. It would be interesting to make comparisons with and without NPT, and IOMMU when it comes out.
HotdogIT - Thursday, July 10, 2008 - link
It's INSANELY difficult to find any good reviews/benchmarks/non vendor-speak about virtualization platforms; looking forward to this article! Even though my virtualization process doesn't involve any heavy duty webservers, seeing how it performs under that load will be cool.Hopefully you'll cover a bit on the install/setup process to; I only mention this because from my experiences, both VMWare's ESX, and Xen's products could both use a bit of spit-shine to the end user, techy or not.
Nehemoth - Friday, July 11, 2008 - link
Yes, And install/setup process of Xen and WMARE ESX would be very helpful.Thank you for the good job.