I got first Sun/MySQL Newsletter Today which among other things lead to the site publishing among other things links to various stuff related to Sun and MySQL and among other things – These Benchmarks
This may be great piece of Benchmarks for Sales and Marketing needs as they show sun stuff is so much cooler but they are so unusable if you really want to take informed decision and pick best of the component level.
As you can see Everything – Hardware, Virtualization, OS, Database Server, Web Server are all different.
This pretty much means you have no clue on what is the best on the component level. May be Sun would stack would get even better results on these Xeons ? or is it “Leading Virtualization Software” used with Windows is a crap ? Remembering horrible MySQL performance loss from Xen and VMWARE It is quite possible. You can same the same about other components.
Looking at container configuration – setting up 6 containers on the same box to run different MySQL and Web instances looks highly dubious to me…. at least this is not what you normally do for high performance Web apps which I have no doubt these benchmarks would be shown as.
Another funny thing is even though title speaks about UltraSparc beating Xeon we get configuration with 48 hard drives which is of course far from what you would see in even typical MySQL DB box, Not mentioning Web box which typically would have very little if any hard drives.
There is also another trap which is hardware/software configuration which is chosen. $200K per box and 830W is probably a lot more than commodity hardware most of us would be using. These days you can get 2*Quad Core Xeon box w 32G RAM and 8 2.5″ hard drives for about 10K from Dell. This would have very close CPU power which what this benchmark tends to compare.
My advice to someone looking at marketing benchmarks would be to examine issues carefully and see what the facts are and which sales decision you’re pushed to using this data. Most likely there will be the trick played with the logic.