September 21, 2014

Another ingenious piece of Sun Marketing

So a while ago I wrote about fun post about MySQL Scalability to 256 way….

Besides discussion on the thread itself I had a lot of private comments in my mail from Sun/MySQL employees which tended to agree with me on this being the a large stretch.

I would hope Sun/MySQL would tone it down and actually spend time on making things to scale inside MySQL rather than asking you to run tens of MySQL instances to get reasonable performance, but sure enough writing code is harder than writing marketing materials so this work now made it to the Sun blueprint making it recommended reference architecture.

It is also interesting to go back to my old post about T2000 performance with Sun comments about Sun not pushing this as scalable MySQL platform. Well this is T5440 now not T2000 but the outstanding gap in single thread CPU performance remains, which makes a lot of operations tasks very complicated.

Oh wait. I forgot there is solution – instead of 1TB database we should run 1000 instances with 1GB each which will allow us not to care about single thread performance for replication needs, ALTER TABLE or any long running queries.

Do not take it as me saying T5440 is never good for MySQL. It may work quite well for some cases, for example when a lot of small MySQL instances need to be run. It is just this blueprint offers very one side view speaking about benefits but no drawbacks.

About Peter Zaitsev

Peter managed the High Performance Group within MySQL until 2006, when he founded Percona. Peter has a Master's Degree in Computer Science and is an expert in database kernels, computer hardware, and application scaling.

Comments

  1. Vadim says:

    What else I would highlight – it may be OK to run N instances on single server, but Sun does not ship management infrastructure for that. You still need to do manual work to clone servers, to setup my.cnf, write custom scripts to deal with start / shutdown, manually create directories and all this error generating actions.

  2. morgan says:

    @vadim:

    Technically, there is mysqld_multi:
    http://dev.mysql.com/doc/refman/5.0/en/mysqld-multi.html

    mysqld_multi was supposed to be replaced by the MySQL Instance Manager. But now the Instance Manager itself is deprecated:
    http://dev.mysql.com/doc/refman/5.0/en/instance-manager.html

    Most people seem to be writing custom shell scripts, so it doesn’t come as a surprise they didn’t use either in their own blueprint.

  3. Honza says:

    This architecture is great for webhosters – they have thousands of small databases.

  4. peter says:

    Vadim,

    This is one issue. The other issue dealing with shared resources complicates performance management dramatically which is important for performance demanding applications. If I have single MySQL application running on dedicated physical box I know hardware capacity and I can relay MySQL activity to activity on hardware side – IOSTAT/MPSTAT etc. With large amount of instances things become much more complicated.

  5. I have read the blueprint a few times just to admire what a brilliant piece of marketing it is. I thought about writing a response, but Peter, when you wrote “writing code is harder than writing marketing materials” you said it so well.

    Someday Percona might get to the point we have the luxury of wasting time and money on things like that blueprint. I hope we never do. I hope we always have more code to write, more people to help, more real value to bring to the community. And I hope we never forget that there are more smart people outside of Percona than there are inside.

  6. Nils says:

    I wonder if this sales pitch actually works, buy the large box so you can do “intra-server scaleout” (instead of just adding capacity as it’s needed)… I still fail to see what Sun’s strategy with MySQL is in terms of creating business and especially synergies.

  7. dim says:

    Folks, I hope you’re smart enough to not generalize an idea about Sun strategy based on a single article.. As many people there are – as many opinions.. That what just one :-))

    Rgds,
    -dim

  8. “Writing code is harder than writing marketing materials” gets quote of the week over here. That’s going up on the wall in the office. :)

  9. William Newton says:

    Yeah. I’m a frequent critic of Sun/Mysql marketing. I just don’t go to their websites unless I absolutely have to these days. I like the product it works well enough, but ads like this are a definite turn off, and devalues any other advice they give. Well, at least when its presented in that format. Mysql and Sun both do have very talented engineers who do offer great advice, but not like that. You have to be careful not to throw out the baby with the bathwater.

  10. David Lutz says:

    Hi Peter,

    I think that this blueprint article shows one example use case for MySQL on a CMT platform, but it certainly isn’t intended to suggest that CMT is a preferred platform for MySQL. Quite the opposite, we regularly tell people that systems with relatively small numbers of fast cores are much more likely to be a good fit for MySQL than highly threaded CMT systems. This CMT use case makes the most sense for installations that are already making substantial use of horizontal or diagonal scaling, and can reduce their cost and complexity with CMT systems. For example, at the high end of these installations, you may see hundreds or even thousands of MySQL instances running on several hundred or even a few thousand systems. In a situation like that, you may be able to reduce the number of servers by an order of magnitude, and see a huge savings in acquisition costs, power and cooling costs, and floor space savings. The point isn’t to say that this is the norm, it is to say that this is an option that will make sense for some people.

  11. peter says:

    David,

    Very good to hear. Your approach makes sense and indeed there is variety of the architectures and hardware solutions which excel in different cases. You have described this as a niche solution and It would be very nice if blueprint would also do that – outlining exact assumptions where such approach makes sense. Such important sections are omitted way to often.

  12. David, I agree with Peter that your approach makes a lot of sense, but Sun doesn’t speak with a single voice here. If you read MrBenchmark’s articles and blog comments in various places, and then read the paper, the take-away is very different from what you’re advocating. I would argue that it DOES suggest that this blueprint is the preferred architecture. But that could be fixed, right?

    It’s also not too late to edit the blueprint and amend the comments about people whom, the authors apparently believe, have been stuck in patterns that their limited intelligence doesn’t let them escape. Those phrases tend to make the whole paper sound arrogant and chest-thumping.

    But it’s just a PDF that can be edited and re-uploaded, if you see what I mean. It might serve Sun better that way.

    MrBenchmark’s comments in the blogosphere are very unlikely to be so easily redacted, though :)

  13. Mark Callaghan says:

    David,

    I don’t think that article does CMT justice. First, for the single system tests the results show linear scale-down (rather than scale-up) for throughput. The test determines the max per-client throughput at high concurrency and uses that to rate limit clients at low concurrency. The same technique can be used to show linear scale-up for just about any server software.

    Second, it shows great throughput on a workload when you shard a database. That is not a good reason to shard a database. The focus of the article should have been on server consolidation for which CMT servers are well suited.

    Finally, if you are pitching big CMT boxes for consolidation, then this isn’t a solution yet. How are we expected to manage this? Running ‘/etc/init.d/mysql start’ separately for each instance doesn’t scale when there are tens or hundreds of instances on a box. What is the management tool? The tool is even more critical when you move to cloud deployments with many big CMT boxes and even more database instances to manage.

  14. Hi all;

    Thanks for your great comments. The intent of my initial blog post as well as the (marketing – YES !) BluePrint is to generate a conversation about MysQL with CMT servers. As far as I am concerned, this is a success. MySQL on CMT is not a Reference Architecture but an available option. Be sure that SUN is very actively writing code to produce a highly scalable engine.
    So, I’d like to propose the following correction for Robert’s wall: “Writing SCALABLE code is harder than writing marketing material”.

  15. @MrBenchmark
    I’ll post that quote as well then get back to writing [scalable] code. :)

  16. Benoit,

    (From now on I won’t call you MrBenchmark). Yes, we know, actively writing a scalable engine. But I have some questions for you.

    Do you understand what Mark’s comments about linear scale-down mean? Do you understand the implications of that? Why don’t you translate Mark’s comments, repost them in your own words, and let’s see what you think it means.

    What is your job, exactly? Is it to generate conversations? I mean, on the one hand you’re putting out this paper, and on the other you’re saying it’s not even meant to be a technical reference, it’s just to start conversations. Are you Sun’s version of SciGen, then? People are still having conversations about SciGen, years later. Success! Are you MrBenchmark or MrConversation?

    Your comments put you and your paper in a new light, which I think is how most people have viewed things all along. Do you think what people are saying in private email conversations about you and your work is a success?

  17. Peace has never come from dropping bombs. Real peace comes from enlightenment and educating people to behave more in a divine manner.
    -Carlos Santana

  18. @Mark,

    “How are we expected to manage this? Running ‘/etc/init.d/mysql start’ separately for each instance doesn’t scale when there are tens or hundreds of instances on a box. What is the management tool?”

    We’ll be publishing the Tungsten Manager in open source later this week–it uses group communications to do broadcast commands to start/stop/restart network services. With large clusters group communications move from the realm of academic curiosity to a very useful tool. Stay tuned…

    Cheers, Robert

  19. Nils says:

    MrBenchmark:
    So why would I choose CMT boxes over standard x86 based systems which run a single instance?

  20. Mark Callaghan says:

    I don’t get commissions on sales of CMT boxes, but I think there is a case to be made for them. My criticism was about the presentation not about CMT. For example, I would like to see performance/watt for throughput oriented benchmarks. I know that single core performance on them won’t match x86 but that should also allow them to be more efficient. Also, some storage engines can take advantage of many-core servers (well only multi-threaded NDB can do this today). Finally, if CMT allows for fast switches between hardware threads on a core then it should provide better utilization for workloads with a lot of memory stalls and DBMS workloads have a lot of them.

    I think that server consolidation and CMT may be a good match as long as there are good management tools.

Speak Your Mind

*