MongoDB Query Plan Cache Explained: Performance, Pitfalls, and Re-Planning

When MongoDB receives a query, it performs the following steps:

• Evaluate the available indexes that could be used.
• Generate and test multiple execution plans using candidate indexes.
• Measure their performance during a trial phase.
• Select the fastest plan (the winning plan) and execute the query.

These steps are known as query planning, and they are expensive in terms of CPU, memory, and potentially disk utilization. Calculating a query plan for every single query would significantly impact database performance.

Fortunately, MongoDB does not repeat this process for every query.

Once a query plan has been generated, it is stored in an in-memory cache associated with the collection: the Query Plan Cache. Each collection has its own independent cache.

The Query Plan Cache stores:

• The query shape (filter conditions, projection, sort, etc.)
• The winning plan (indexes and access methods)
• Additional metadata, such as the expected performance of the plan (measured in works, an abstract unit of measurement that MongoDB uses internally to quantify the resource consumption of a query execution plan)

Every subsequent query with the same shape retrieves the winning plan directly from the cache instead of triggering query planning again.

• Benefits are:
• Less CPU utilization
• Reduced latencyFaster recurring queries
• Better overall scalability

Let’s test it

Let’s explore the Query Plan Cache and see how to inspect and manage it.

Create a Test Collection

Create an orders collection with 1 million documents using the following script in the shell:

Create Indexes

Create secondary indexes, so MongoDB can evaluate multiple execution plans:

Generate and Inspect a Cache Plan

Run the query a few times to ensure the plan is cached:

Inspect the execution plan:

Relevant details:

The winning plan:

• is cached
• uses the compound index
• shows good execution metrics

The Query Plan Cache is now populated with this optimal plan.

Inspecting the Query Plan Cache

You can inspect the cache using :

Alternatively, via aggregation:

From the cache, you can see:

• Unique identifiers for the query shape (queryHash, planCacheKey)
• The cached execution plan (cachedPlan)
• The expected cost (works)

Reusing the Cached Plan

Now run the same query shape with different values:

From explain(), you will see:

This confirms the plan was reused from the cache (isCached: true and look at the referenced planCacheKey).

Verifying Cache Usage with the Profiler

If the profiler is enabled, you can check into the system.profile collection of your database:

This also confirms that the cached plan was used (fromPlanCache: true and look at the referenced planCacheKey).

When the Cached Plan becomes Suboptimal

Let’s degrade the selectivity by setting all orders to status=’PAID’.

Now status is no longer selective.

Running the original query again:

You may notice:

• Higher keysExamined
• Increased executionTimeMillis

MongoDB is still using the cached plan, even though data distribution has changed.

This could be the main concern about the cached query plans. If the data distribution changes without being so bad, MongoDB continues to use that plan. This way it could happen that the cached plan is no longer the best one.

Manually clearing the cache

Clear all cached plans and check it is really empty:

Or remove a specific plan by providing the query shape:

Query Plan Cache invalidation

Since the data changes over the time, the query plan for a specific query can change as well. The winning plan depends on:

• Available indexes
• Data distribution.

Doing massive inserts, updates or deletions can alter the distribution of the data and a plan that was optimal yesterday, cannot be optimal today. This means the query plan needs to be recalculated from time to time to make sure MongoDB can rely on the most up-to-date and optimized one.

Plans are recalculated only when invalidated, not on a schedule.

Invalidation happens when:

• An index is created
• An index is dropped
• mongod restarts
• The cached plan becomes inefficient. MongoDB compares actual performance with expected performance (works). If the deviation is large (different order of magnitude), the plan is invalidated and replanned

The Re-Planning issue

Everything works well until you face the re-planning issue.

It could happen that every query triggers the invalidation of the cached plan, and so a re-planning is needed continuously. You can easily identify when this happens because usually you can notice a significant CPU utilization increase.

You can also verify in the mongod log what is happening:

If you see fields like “replanned”:true, and “replanReason”:”cached plan was less efficient than expected: expected trial execution to take 2 works but it took at least 20 works”, it means the query required re-planning.

Look at the “planningTimeMicros”:215107053 and compare it with “durationMillis”:215107; almost the entire time was spent in the query planning.

Usually, a query replanned is also a slow query. Then the occurrences are easy to identify by filtering the log with “id”:51803. This is the unique identifier for all slow query entries in the log.

Common causes of continuous re-planning:

• Having unstable queries that are data-dependent
  • Indexed values are not evenly distributed: a plan is good for specific values of the filter conditions and not for others
• Having right indexes but not selective enough
• There are 2 or more plans providing the same performance. Little variations in the data can change the real performance of the queries
• The cardinality of indexed fields changes rapidly. This could happen when:
  • Doing massive inserts
  • Doing massive deletes
  • Having TTL indexes that delete lots of documents
  • Using capped collection that overwrites lots of documents
• Using $or, $in or $regex operators can lead to completely different results depending on the number of items you specify in the query

Write-intensive workloads are generally more sensitive than read-intensive ones.

In short: the re-planning loop occurs when data changes faster than the planner can adapt.

Mitigating Re-Planning

There is no universal fix, but you can

• Analyze the affected query and create more specific and selective indexes
• Avoid overly generic queries that can be affected by the uneven data distribution
• Reduce the number of items when using $or or $in operators. Eventually split the query in multiple smaller queries
• Regularly analyze explain(“executionStats”) for monitoring the execution plan
• Use hint() to force the utilization of a specific index in extreme cases.
  • Using hint() in production is not ideal, but in pathological situations it can immediately stabilize performance while you investigate properly.

About hint(), there is a new feature available in MongoDB 8.0 that provides a way to hint a specific index for any query shape for the entire cluster. More details on this page.

Query Plan Cache is Local

Important to remember is that the Query Plan Cache is:

• Per collection
• Local to each mongod instance

There is no cluster-wide global cache.

In a sharded cluster:

• Each shard may have a different plan for every query
• PRIMARY and SECONDARY nodes may differ as well if you created different indexes
• Chunk migrations can influence the plan selection

Starting from version 6.3 it is possible configuring the size of the Query Plan Cache using the parameter planCacheSize. It can be set to a percentage of available memory like for example “8.5%” or to a fixed size in MB or GB, like for example  “100MB” or “1GB”. It defaults to 5%.

For more details look at this page.

Conclusion

The Query Plan Cache is a powerful MongoDB feature that stabilizes and improves query performance by avoiding repeated planning. This also helps to increase scalability.

However, in specific scenarios—especially with unstable data distribution or low-selectivity indexes—the re-planning loop can cause high CPU usage and degraded performance.

Understanding how the cache works, how invalidation happens, and how to detect re-planning issues is essential to maintaining predictable and efficient query performance.