How to shard effectively
Sharding a database is hard. MongoDB folks will tell you they have already done it for you. And they have sort of. In the MySQL world Oracle via MySQL fabric and Scale Arc will tell you that they have done it for you. Cassandra folks will tell you they have avoided the need all together. What they have all done is provide tools that take care of some of the more easily automated work associated with sharding. They have not done the hard parts. In this talk we will talk about the hard parts. - What kinds of data shard easily and what kinds do not - What will you use as your sharding methodology - How sharding impacts your data model - How sharding affects each query you write - How do you manage your sharding scheme over time as your database grows or shrinks We will also talk about what MongoDB chunking, Cassandra partitioning and MySQL Fabric or Scale Arc will do for you and how they can fit into your sharding scheme.
Principal Consultant, The Pythian Group Inc
In the last 40 years I Have designed and built databases less than a megabyte small enough to fit on cell phones, to databases several petabytes in size, requiring a good sized data center to hold them. Have worked with Many open source database technologies including MySQL, SQLite, MongoDB, CouchDB, CouchBase, PostgreSQL and many others.