Why Optimization derived_merge can Break Your Queries

MySQL optimizer bugsLately, I worked on several queries which started returning wrong results after upgrading MySQL Server to version 5.7 The reason for the failure was derived merge optimization which is one of the default optimizer_switch  options. Issues were solved, though at the price of performance, when we turned it OFF . But, more importantly, we could not predict if any other query would start returning incorrect data, to allow us to fix the application before it was too late. Therefore I tried to find reasons why derived_merge  can fail.

Analyzing the problem

In the first run, we turned SQL Mode ONLY_FULL_GROUP_BY on, and this removed most of the problematic queries. That said, few of the queries that were successfully working with ONLY_FULL_GROUP_BY  were affected.

A quick search in the MySQL bugs database gave me a not-so-short list of open bugs:

At first glance, the reported queries do not follow any pattern, and we still cannot quickly identify which would break and which would not.

Then I took a second look by running all of the provided test cases in my environment and found that for four bugs, the optimizer rewrote the query. For three of the bugs, it rewrote in both 5.7 and 8.0, and one case it rewrote in 8.0 only.

The remaining three buggy queries (Bug #85117, Bug #91418, Bug #91878) have things in common. Let’s first look at them:

  1. Bug #85117
  2. Bug #91418
  3. Bug #91878

Two of the queries use DISTINCT  or GROUP BY , one uses ORDER BY  clause. The cases do not have not the same clause in common—which is what I’d expect to see—and so, surprisingly, these are not the cause of the failure. However, all three queries use generated values: a constant in the first one; UUID()  and COUNT()  functions in the second and third respectively. This similarity is something we need to investigate.

To find out why derived_merge  might work incorrectly for these queries we need to understand how this optimization works and why it was introduced.

The intent behind derived_merge

First I recommend checking the official MySQL User Reference Manual and MariaDB knowledge base. It is correct to use both manuals: even if low-level implementations vary, the high-level architecture and the purpose of this optimization are the same.

In short: derived_merge  is used for queries that have subqueries in the  FROM  clause,  also called “derived tables” and practically converts them into JOIN queries. This optimization allows avoiding unnecessary materialization (creating internal temporary tables to hold results). Virtually this is the same thing as a manual rewrite of a query with a subquery into a query that has JOIN clause(s) only. The only difference is that when we rewrite queries manually, we can compare the expected and actual result, then adjust the resulting query if needed. The MySQL optimizer has to do a correct rewrite at the first attempt. And sometimes this effort fails.

Let’s check why this happens for these particular queries, reported in the MySQL Bugs Database.

Case Study 1: a Query from Bug #85117

Original query

was rewritten to:

You can always find a query that the optimizer converts the original one to in the SHOW WARNINGS output following EXPLAIN [EXTENDED] for the query.

In this case, the original query asks to return all rows from the table table1 , but selects only the generated field from the subquery. The subquery selects the only row with table1id=1 .

Avoiding derived merge optimization is practically the same as joining table table1 with a table with one row. You can see how it works in this code snippet:

However, when the optimizer uses derived-merge optimization, it completely ignores the fact that the resulting table has one row, and that the calculated value would be either NULL  or 1 depending if a row corresponding to table1  exists in the table. That it prints select 1 AS `sel`  in the EXPLAIN  output while uses select NULL AS `sel`  does not change anything: both are wrong. The correct query without a subquery should look like: