A Micro Service Data Layer Discovery Pattern using MySQL, MariaDB MaxScale & Consul
5 October 3:10 PM - 3:35 PM @ Zürich 1
25 minutes conference
The shift to a Microservices oriented architecture for a large organization presents a challenge to manage scale, changes in replication topology and keeping the applications requiring to follow these changes pointing to the correct data sources for reading and writing data. I would like to present a design pattern for Micro Service data layer discovery that we have successfully deployed at Rentalcars.com. Historically we have been using DNS to control what other internal applications and databases are required by a given application. This new pattern solves a few problems, such as: * Waiting for DNS TTL's to expire before applications can failover. (Sometimes these take several minutes, depending on the level of DNS caching). * Moving away from managing complexity around managing multiple SRV records. * There is a finite time before a DNS change would propagate between DNS Zone master to slave servers, this adds to service recovery times. * DNS is a flat structure within a zone, no easy way to create a hierarchy of nodes without using sub-domains. * Maxscale deployed as a load balancer in HA pairs, getting over-whelmed by the traffic through them at times. * Integrate a topology manager like the Outbrain Orchestrator into the above solution. What we were trying to achieve in addition to that was: * Increase the number of Maxscale instances while gaining better control over when configuration changes get pushed out. * Add a distributed key/value store to the solution that lets us provide immediate switchovers for above, in a hierarchy that is flexible. * Have a DNS service that fails over faster, ideally feeding off the same Key/value store. * Let applications discover through the Key/value store what backends they need to connect to. * Move the load balancing element (Maxscale) closer to the application servers so it becomes independent of the 2-node HA pair becoming a bottleneck. * In-case the MySQL topology grows/shrinks, the Maxscale configuration should keep up with the changes. We would present the above ideas, how the solution evolved and what results were achieved.
Principal Database Engineer, Rentalcars.com
Subject matter expert at Rentalcars.com for both SQL and NoSQL technologies.