September 23, 2014

MySQL 5.6: Improvements in the Nutshell

Preparing for my talk for Percona MySQL University in Raleigh,NC, Tuesday 29th of January I have created the outline of improvements available in MySQL 5.6 which I thought was worth sharing to give a feel for how massive work have been done for this release in variety of areas. I’m sure the list is not complete so If I miss something significant please let me know through the comment and I’ll update the page

Scalability
– Scalable Read Only Transactions
– Concurrent Innodb data file extension
– Non-Recursive Deadlock Detection
– Faster Locking Primitives
– Improved Innodb Thread Concurrency
– Multiple background Purge Threads
– Improved Purge lag control (now works)
– Split of “Kernel Mutex”
– Data Dictionary Cache
– Improved Adaptive Flushing
– Page Cleaner/Separate Flush Thread
– Group Commit for Binary Log
– Fight Cache Coherence and False Sharing issues
– Reduced Innodb Memory Fragmentation
– Reduced Locking for Partitioned tables
– Reduced Contention for LOCK_open
– Support for multiple table_open_cache instances
– Large (over 4GB) redo logs support

Optimizer and Execution
– Index Condition pushdown (ICP)
– Multi-Range-Read (MRR)
– Faster ORDER BY nidxcol LIMIT N
– Persistent Statistics for Innodb
– Improvements to Innodb Compression
– Fast Page Checksums (CRC32)
– 4K and 8K Page sizes for Innodb
– Improvement to Buffer Pool Flushing
– Subquery Optimizations
– More efficient Optimizer

Replication
– Optimized ROW Based Replication
– Multi-Threaded Slave
– Global Transaction Identifiers
– Crash Safe Slave and Binlog
– Replication Event Checksums
– Time Delayed Replication
– Server UUID
– Improved Logging for Row based Replication
– Replication Utilities for Failover and Admin

Transparency
– Many new INFORMATION_SCHEMA Tables
– – INNODB_METRICS
– – Meta Data Information Tables
– – Buffer Pool Information Tables
– Improved PERFORMANCE_SCHEMA
– – Reduced Overhead
– – Simplified Configuration
– – Table Access instrumentation
– – Statements instrumentation
– – Stages Instrumentation
– – Aggregations by User, Host etc
– – Network IO Instrumentation
– – Show Host Cache Contents
– – Improved File I/O Instrumentation
– Improved EXPLAIN
– – Explain for UPDATE/DELETE queries
– – JSON output with more information
– Optimizer Tracing
– Deadlock Logging
– GET DIAGNOSTICS

Operational Improvements
– Separate Tablespaces for Innodb Undo Logs
– Fast Restart – Preloading Innodb Buffer Pool
– Online DDL
– Import/Export for Partitioned Tables
– Remote Binlog Backup
– Innodb Transportable Tablespaces
– New configuration files defaults
– User Defined DATA DIRECTORY for Innodb Tables
– Connection Attributes

New Functionality for Developers
– MemcacheD API in Innodb
– Explicit Partition Selection in queries
– Full Text Search index for Innodb
– Microsecond TIME precision
– Precise spatial operations in GIS

Security
– Password hashes instead of plain passwords in Query Logs
– SHA256 hashing with Salt for Authentication
– Use obfuscated password storage for command line tools
– Policy Based password validation
– Plugin authentication support for Replication

About Peter Zaitsev

Peter managed the High Performance Group within MySQL until 2006, when he founded Percona. Peter has a Master's Degree in Computer Science and is an expert in database kernels, computer hardware, and application scaling.

Comments

  1. some more – connection attributes, expire password at next login.

  2. Marc Alff says:

    The content of the host cache is also visible in the performance_schema now.

    Configuration of the performance_schema is easier, with many sizing parameters that can be sized automatically.

  3. * User defined DATA DIRECTORY per InnoDB table
    * Reduced partition locks (only locks relevant partition as opposed to going through the locking mechanism of all partitions)

  4. marc castrovinci says:

    Can’t remember the last time I was so excited for a release. I’m really looking forward to the optimized order by/limit and GTID.

  5. Thank you for comments. I’ve now updated the post with your comments.
    The password expiration is one of the features of password policies.

  6. I tried to get “Obfuscated passwords in Query Logs” to work today. But it didn’t appear to work.

    http://bugs.mysql.com/bug.php?id=68200

  7. Daniel,

    Thanks for comments. Well it brings the question of cases are covered about it. The feature is mentioned for example here:
    https://blogs.oracle.com/MySQL/entry/mysql_5_6_is_a

    I did not dig much further into it as there is a little value in security by obfuscation – I expect the “password recovery tools” will appear for MySQL rather shortly

  8. Dimitri says:

    Peter, more about performance/scalability:
    – improved adaptive flushing (really improved + transparent for tuning and monitoring)
    – fixed purge lag tuning (no more danger, designed right now + transparent, I’ll yet blog about)
    – fixed LOCK_open contentions and introduced table open cache instances
    – over 4GB REDO logs
    – FILE I/O instrumentation in Performance Schema (not only waits and access, but also TIME of I/O operations!)

    yes, the list is really huge, pretty sure we’ve still missed something ;-)

    Rgds,
    -Dimitri

  9. Thanks Dimitri,

    I have added your comments. I think purge lag tuning is more of the bug fix than new feature… though you can see the Group Commit as such too.

  10. Dimitri says:

    Hi Peter,

    stay tuned, I hope purge lag tuning will see a new life in 5.6,
    let’s see.. ;-)

    Rgds,
    -Dimitri

  11. Peter,

    If I understand it correctly this isn’t just obfuscation. Instead of logging PASSWORD(‘password’) the resulting (one-way) hash will be logged.

    This could be done for the binlog manualy by doing
    SET @pwd := PASSWORD(‘password’);
    INSERT INTO… VALUES(@pwd)

    The new feature should make this automatic and will be applied to all logging options.

  12. Daniel,

    Thanks for catching it. Indeed it is perhaps different for logging and the command line password, which is indeed obfuscated. For logging of password we can indeed just log one way hash

  13. aftab says:
  14. marc castrovinci says:

    Daniël,

    Nice trick for keeping the password out of the log.

    It will end up in .mysql_history though, which is always an area of concern. If an intruder did get on the DB server, there is no difference of gaining the password from log files or history files.

    Really need to write a script to scrub out the .mysql_history file in addition.

  15. Marc,

    A solution for this issue is in place too.
    Check this out :
    http://bugs.mysql.com/bug.php?id=48287

  16. Is the password in the log just the hash from the user table? If so, you can modify the MySQL client to salt that hash and it is just as bad having the hashed password as the original. Just like the “encryption” in the new command line utils which is no better than ROT13.

    btw, innodb_autoextend_increment affects innodb_file_per_table tablespaces. I don’t see that on the list.

Speak Your Mind

*