Thinking more about the problems I wrote about yesterday I had a question why so ugly workaround and guesses or manual configuration is needed ? The answer seems to be Operation Interfaces just do not provide IO interface which is good enough. The big missing piece is priority. There are process and threads priorities in most operation systems but there is no priority you can set for given IO request. In fact in most cases even thread priority is not accounted for while executing IO request. Last time I checked there was IO scheduler with these functions in works for Linux Kernel but I have not seen anything production ready.
So what I would like to see ? Each IO request should be able to be assigned priority, at least from 3 hard classes – “RealTime”, “Normal” and “Idle”. More fine grain control is good but not necessary. What databases could do if such interface would exist ?
The other concept which would be quite hepful (for asynchronous IO) is priority escalation. Aternatively you could of course simply cancel old requests and submit them with new priorities but it is ugly. This is needed as in certain cases load will be high enough so idle bandwidth will not be enough for flushing dirty buffers or keeping up with checkpointing. In this case you would like to raise priority of these activities but as a lot of IO requests already submited it should be good to increase priority of these.
I would be very interesting to hear if there is any development going on for something like this in modern operation systems.
In fact these are not new ideas. I’ve been working with Philippe Bonnet students from “Badger” projects at DIKU We tested some protototype patches for Linux kernel and MySQL but we did not get anything ready for production up to this point.