MongoDB Multi-Document Transaction Fails When getLastErrorDefaults is Changed

In every new version of MongoDB, there have been a lot of changes and newly introduced features. One such change is the introduction of setDefaultRWConcern command from MongoDB 4.4. This feature has caused multi-document transaction writes to fail for one of my customers. In this blog post, we will look into the problem and how to resolve it.

Introduction

When you want to set the default common writeConcern for your replicaSet to use if writeConcern is not specified explicitly in the command, then you can set it via rs.conf().settings.getLastErrorDefaults values. The default value is {w: 1, wtimeout: 0} i.e. requires an acknowledgment from PRIMARY member alone and this has been there for a long time in MongoDB. After upgrading to MongoDB 4.4, the customer was facing issues with multi-document transactions and the writes failed causing application downtime. 

Issue

The cluster/replicaset wide writeConcern could be changed via rs.conf().settings.getLastErrorDefaults value till MongoDB 4.2. But from MongoDB 4.4, if you are using a multi-document transaction with the non-default values of getLastErrorDefaults, then it will eventually fail with the below message:

This is because changing the default value of getLastErrorDefaults was deprecated from MongoDB v4.4 and to change the global writeConcern/readConcern, it needs to be done via the new method – setDefaultRWConcern which was introduced in MongoDB 4.4.

Test Case

The default value of getLastErrorDefaults is {w: 1, wtimeout: 0}. Let’s change it to different values – { “w” : “majority”, “wtimeout” : 3600 }:

When running a transaction into a collection percona.people, the error occurs and complains that writes could not be written with the changed getLastErrorDefaults value:

How to Resolve It

To resolve this issue, revert getLastErrorDefaults to its default value if changed. Let’s change it to the default value as follows:

Then mention the required default writeConcern, readConcern via the command setDefaultRWConcern as follows:

Then write with a transaction into the collection percona.people:

Check the data with the different sessions to validate the writes:

Note:

Here note that normal writes (non-transaction writes) are not affected by changing defaults for getLastErrorDefaults. So if you have an application that doesn’t use transactions, then you don’t need to act immediately to remove non-default values in getLastErrorDefaults though it is not recommended. 

How setDefaultRWConcern Works

Here, we will also verify a case, whether the default writeConcern set via setDefaultRWConcern is working as expected. For testing, let’s take down a member from the 3 nodes replicaSet:

Then test it via a insert command with writeConcern mentioned in it explicitly. It should fail as there are only 2 members alive and {w:3} needs acknowledgement from 3 members of the replicaSet:

Let’s now test by setting setDefaultRWConcern to { “w” : 3, “wtimeout”: 30 } as follows:

Now try the write within a transaction or normally without specifying the writeConcern explicitly:

With Transaction, the error occurs on commitTransaction() as follows:

How to Check DefaultRWConcern

You can get the current value of read/write concerns via getDefaultRWConcern command.

Conclusion

From 4.4.0, the workaround for the said problem is to use the defaults for getLastErrorDefaults until 4.4.6 and if needed after 4.4.6, set the cluster/replicaset wide write/read concerns via setDefaultRWConcern command instead. The version MongoDB 4.4 honors any write concern value that you specify in getLastErrorDefaults, however, it is not allowable from v5.0 (refer SERVER-56241). This behavior was reported in JIRA SERVER-54896, SERVER-55701 and got fixed from the version 4.4.7 to ignore the value of getLastErrorDefaults as per the bug fix. 

Also, when you do MongoDB 4.4 upgrade, you can note this and if needed change your default read/write concern settings through setDefaultRWConcern method.

Hope this helps you!

Percona Distribution for MongoDB is a freely available MongoDB database alternative, giving you a single solution that combines the best and most important enterprise components from the open source community, designed and tested to work together.

Download Percona Distribution for MongoDB Today!