Thinking more about the problems I wrote about yesterday I had a question why so ugly workaround and guesses or manual configuration is needed ? The answer seems to be Operation Interfaces just do not provide IO interface which is good enough. The big missing piece is priority. There are process and threads priorities in most operation systems but there is no priority you can set for given IO request. In fact in most cases even thread priority is not accounted for while executing IO request. Last time I checked there was IO scheduler with these functions in works for Linux Kernel but I have not seen anything production ready.
So what I would like to see ? Each IO request should be able to be assigned priority, at least from 3 hard classes – “RealTime”, “Normal” and “Idle”. More fine grain control is good but not necessary. What databases could do if such interface would exist ?
The other concept which would be quite hepful (for asynchronous IO) is priority escalation. Aternatively you could of course simply cancel old requests and submit them with new priorities but it is ugly. This is needed as in certain cases load will be high enough so idle bandwidth will not be enough for flushing dirty buffers or keeping up with checkpointing. In this case you would like to raise priority of these activities but as a lot of IO requests already submited it should be good to increase priority of these.
I would be very interesting to hear if there is any development going on for something like this in modern operation systems.
In fact these are not new ideas. I’ve been working with Philippe Bonnet students from “Badger” projects at DIKU We tested some protototype patches for Linux kernel and MySQL but we did not get anything ready for production up to this point.
Percona’s widely read Percona Data Performance blog highlights our expertise in enterprise-class software, support, consulting and managed services solutions for both MySQL® and MongoDB® across traditional and cloud-based platforms. The decades of experience represented by our consultants is found daily in numerous and relevant blog posts.
Besides specific database help, the blog also provides notices on upcoming events and webinars.
Want to get weekly updates listing the latest blog posts? Subscribe to our blog now! Submit your email address below.