128GB or RAM finally got cheap

PREVIOUS POST
NEXT POST

I did not usually go to “Elite” servers on Dell web site but looking at customers system today I went to check Dell Poweredge R900. This monster takes up to 4 Quad Core CPUs and has 32 memory slots, which allows to get 128GB of memory with 4GB of memory chips. This means upgrade to default configuration to 128GB of memory will cost you just $9600 (list price). I’ve been able to configure on a web the system with 8*2.5″ hard drives RAID and 2 CPUs (just as we usually configure PowerEdge 2950) with 128GB of RAM for about $16000. This means talking to Dell Sales rep it can purchases within $15000. This may sounds as a lot but if you’re memory constrained it is cheaper per GB than buying 32GB box for $6000

So am I scale-up advocate ? No. But it is quite frequently systems are designed to have “working set to fit in memory” to perform well and such systems can avoid good consolidation factor for such application, or would allow them to delay sharding.

This box also has 4 CPU sockets which means 16 fast cores and 128GB of memory becomes commodity – Quite a challenge for MySQL to take :)

I have not had a chance to play with such box myself besides couple of customer production installations but it looks pretty sweet.

PREVIOUS POST
NEXT POST

Comments

  1. says

    Is that memory usable with MySQL, or do you use if for a large OS buffer cache? 128GB in the MyISAM key cache protected by one mutex doesn’t sound like a winner. I am not sure if 128GB in the InnoDB buffer cache protected at times by one mutex would be much better.

  2. says

    Mark,

    In the cases I’ve seen it was actually used for Innodb buffer pool. 100GB Buffer pool this is sweet (though counting pages for SHOW STATUS make it to take over 1 sec).

    But anyway we should not mix concurrency issues with cache hit ratio. If you’re consolidating 4 32GB boxes to 1 128GB box you will need to ensure the multiplied concurrent load can run it.
    If you’re just getting more memory to stop falling off the cliff because of IO bound the fact there can be big fat mutex may not be stopping you. It is still much faster than disk.

    And well… we should fix Innodb so there is no single mutex used :)

  3. Justin says

    How did you get it down to $16k? when i pick 2xQuad Core and only a single
    hard drive, and no OS etc, and 128gb of memory, it comes to $22,352

  4. Justin says

    never mind, the way dell offers prices to some customers and not others is confusing! you have to go in via “small and medium business” to get the price.

  5. says

    There are also 8 CPU Opteron boxes (that’s 32 cores!) which support up to 256GB of RAM, but you’ll usually end up at around 45k $, not really commodity but still a lot less than large Sparc or Itanium Boxes.

  6. says

    Honza,

    Sure. In world of commodity if Dell has systems everybody has similar systems too. I note Dell in particular because we deal a lot with Dell hardware and the fact they tend to ship relatively mature platforms. My experience certain new systems from SuperMicro and Tyan was not always positive.

  7. says

    We’ve got a pair of HP DL580 G5 servers in Active/Passive configuration with a similar configuration to the Dell you mention. Two quad core CPU’s and 128GB of RAM (in 16 8GB sticks.. so about $25k instead of $10k). It also has 14 SAS disks in it, with two 512MB Battery Backed Write caches setup in a RAID 5+0 configuration. I have it configured with large_pages and about 80GB in the innodb_buffer_pool. The rest is reserved for temp tables (20GB tmpfs partition) and connections (large buffer sizes are setup).

    The only thing we’ve found is that it really will only use 2 – 3 of those 8 cores.. this is probably because of innodb’s threading issues. However.. having that much RAM means *zero* IO bound load. And even when we do need to hit the disks, having that much write cache means that the system never shows more than about 2% “io wait” (and thats 2% of a possible 800%…). If it weren’t for next-key locking in big selects, the system would appear nearly transparent. ;)

    Its definitely not ideal. Apps should be designed to scale out. However, in this case, being able to quickly insert many thousands of records per second into one DB, and then have them fan out (via various methods) to the parts of the application that must scale out, means the critical parts of the app can be simpler and (theoretically) have less bugs.

  8. says

    Thanks for feedback Clint. Indeed Innodb has issues depending on workload and it becomes more and more problems as more and more cores are becoming commodity.

    Regarding Scale Out – true though Scale Out should allow you to pick the optimal size of the box. For most application this optimal size is pretty large because there is operations/monitoring etc overhead involved with each not to mention space/power concerns which are the often the reason for consolidation.

    It on large scale app I would rather have 10 128GB boxes than 50 of 16GB boxes to deal with :)

  9. Matt says

    We have two of the big Dells and mostly love them, so far at least. Don’t know what we’d do without them. Memory I/O is no issue, everything is very fast to our surprise.

    We have encountered some issues with swapping when all 128G is filled (we do this a lot); the whole system can hang up completely, this may be a disk controller issue.

    The big advantage of the Dell is that it will take 128G in 4G sticks. Very few other machines will do this currently. 8G sticks more than double the total cost. I believe it is based on a chassis that Intel produce.

    If there was a machine that took 256G in 4G sticks I would buy one tomorrow.

  10. Tao says

    sunfire x4600 is a 8 socket 8 dimms per socket Barcelona opteron system. It will fit 256gb ram on 64 total simms

  11. Nicolai Plum says

    The Sun X4600 (http://www.sun.com/servers/x64/x4600/specs.xml) will take up to 512MB RAM (though you will likely need to put 8 CPUs in it to get enough memory controllers to address all those DIMMs). Commensurately if contributor “Matt” has one with those 8 CPUs, it would be possible for them to install 64 x 4GB DIMMs and get the desired 256MB in 4GB DIMMs.

Leave a Reply

Your email address will not be published. Required fields are marked *