October 24, 2014

A piece of Sun/MySQL Marketing

I got first Sun/MySQL Newsletter Today which among other things lead to the site publishing among other things links to various stuff related to Sun and MySQL and among other things – These Benchmarks

This may be great piece of Benchmarks for Sales and Marketing needs as they show sun stuff is so much cooler but they are so unusable if you really want to take informed decision and pick best of the component level.

As you can see Everything – Hardware, Virtualization, OS, Database Server, Web Server are all different.
This pretty much means you have no clue on what is the best on the component level. May be Sun would stack would get even better results on these Xeons ? or is it “Leading Virtualization Software” used with Windows is a crap ? Remembering horrible MySQL performance loss from Xen and VMWARE It is quite possible. You can same the same about other components.

Looking at container configuration – setting up 6 containers on the same box to run different MySQL and Web instances looks highly dubious to me…. at least this is not what you normally do for high performance Web apps which I have no doubt these benchmarks would be shown as.

Another funny thing is even though title speaks about UltraSparc beating Xeon we get configuration with 48 hard drives which is of course far from what you would see in even typical MySQL DB box, Not mentioning Web box which typically would have very little if any hard drives.

There is also another trap which is hardware/software configuration which is chosen. $200K per box and 830W is probably a lot more than commodity hardware most of us would be using. These days you can get 2*Quad Core Xeon box w 32G RAM and 8 2.5″ hard drives for about 10K from Dell. This would have very close CPU power which what this benchmark tends to compare.

My advice to someone looking at marketing benchmarks would be to examine issues carefully and see what the facts are and which sales decision you’re pushed to using this data. Most likely there will be the trick played with the logic.

About Peter Zaitsev

Peter managed the High Performance Group within MySQL until 2006, when he founded Percona. Peter has a Master's Degree in Computer Science and is an expert in database kernels, computer hardware, and application scaling.

Comments

  1. Welcome back. Where is the specification for the iGen benchmark?

  2. peter says:

    Thanks Mark,
    Things are still quite erratic for me – still slipping on inflatable bed etc.
    Right the fact there is not link to iGen benchmark specs or sources is another issue I should have noted :)

  3. Bill says:

    Marketing benchmarks like those are a prime reason why sites like yours ( and the mysql community in general) are so valuable. Thanks again for you independent perspective on all things mysql.

  4. Ritu Kamboj says:

    First of all, let me take this opportunity to thank you for taking the time to review the benchmark.

    This benchmark was about open stack vs. Microsoft stack w/virtualization.
    It was not meant to be determining specific effects of each part of the
    stack. Prix fixe menu vs. ala carte.

    Current virtualization/consolidation that many customers are doing combine
    a lot more than 6 different workloads and that trend is continuing.
    Consolidating more low-utilization workloads together can save a lot more
    wattage than many servers running at low utilization.

    The iGen workload was created from actual customer workloads and has a lot
    more complexity than Sysbench which only test very simple operations one
    at a time. The iGen database consist of 6 tables and its executes a combination of light , medium and heavy transactions. The disks was appropriately sized for this workload. We realize that typical MySQL DB boxes may be running lighter workloads that require less disks. Many workloads on Oracle, DB2, and Sybase workloads require a lot more disks.

    The $200k was for hardware, disk, Microsoft, and virtualizaiton software,
    not just a box.

    There are many benchmarks that a UltraSPARC T2 configuration can beat
    4-socket quad-core Xeon both on price, performance, price/performance,
    watt/performance when you have workloads that need this level of computation. Check this out:

    http://www.sun.com/servers/coolthreads/t5220/benchmarks.jsp
    http://www.sun.com/servers/coolthreads/t5120/benchmarks.jsp

    While not all customers require Hardware, disk, and software configurations
    of this size there are many workloads that require a lot more. It would be
    interesting to see how MySQL scales to address these even larger requirements.
    It may present a growth area for MySQL.

    You can get these servers for free 60 days trial and check them out for consolidation with free Solaris containers technology of Solaris 10. Check this out for more details:
    http://www.sun.com/tryandbuy/rules.jsp.

    If availability of disk is an issue, maybe you can use another workload like Sysbench instead of iGen

  5. peter says:

    Ritu,

    Honestly why I picked up on this is because of the title which says “MySQL Benchmark UltraSPARC T2 beats Xeon on Consolidation of OLTP & Web” – it does not Say Sun Open Source Stack beats Microsoft Stack it speaks about CPUs.

    Indeed this is full stack benchmark and what I’m saying it is at large extent pointless because there are so many variables involved. Again I’m speaking from the true technical standpoint.

    Regarding the benchmark I hope you understand “mix of simple medium and heavy transactions” does not count as a good benchmark specs. The only onces which allow to repeat benchmark are good.

  6. Ritu,

    Are iGen benchmarks going to be published comparing Falcon with InnoDB? There might be a negative reaction from part of the MySQL community if that were done without also publishing a spec for the iGen benchmark.

Speak Your Mind

*