Permissions Issue When Upgrading MongoDB with Custom dbPath or logPath

mongodb permissions issueIn Linux distributions, installing MongoDB for the first time using rpm/dpkg will create a conf file, dbPath, logPath and the init scripts on default paths. This includes the creation of a mongod user/group and provides mongod permissions to both the default dbPath (/var/lib/mongo) and logPath (/var/log/mongodb/mongod.log). Then you edit the settings in the config file at /etc/mongod.conf for custom settings like dbPath in different locations and start mongod service as you need.

But this is a little different when you are doing an upgrade or changing between Percona Server for MongoDB (PSMDB) and upstream MongoDB as they include the removal of the existing package and install of the new version. This article explains the problem in some cases with restored user mongod permissions when you are using custom dbPath or logPath locations when reinstalling the packages, and how to overcome it.

Behavior When Upgrading MongoDB Package

We know that uninstalling the MongoDB package would remove the related packages. But especially when you are uninstalling PSMDB, the user mongod is also dropped along with the packages which cause the existing directories to lose permissions and it is left with the uid/gid of the dropped mongod user/group as shown below.

Before Uninstall:

Now remove the MongoDB Package:

After removing the packages, check the user permissions for the mongodb files. They are now assigned with 996:993 which was uid:gid of the dropped mongod user/group:

And when you are reinstalling the PSMDB or upstream MongoDB packages, it would install the packages and create the default directories and mongod user again if not available:

Now check the permissions of the db files on default dbpath:

Default Behavior of MongoDB Installation

If we look into the source code, the installation of the MongoDB package through rpm or through yum/apt-get repository creates the mongod user/group as system user/group as mentioned with SYS_UID_MIN/SYS_UID_MAX and SYS_GID_MIN/SYS_GID_MAX variables (for CentOS, the variables could be found in /etc/login.defs file). The MongoDB code related to it is shown below:

After that, the MongoDB-related directories and files are created and given with the mongod user permissions, as mentioned in the below code snippet, to the default ones. This applies to both PSMDB and upstream MongoDB installations.

Upstream MongoDB Code:

https://github.com/mongodb/mongo/blob/master/rpm/mongodb-org.spec

PSMDB Code:

https://github.com/percona/percona-server-mongodb/blob/master/percona-packaging/redhat/percona-server-mongodb.spec.template

Issue Occurs When Using Custom db/log file Paths:

When you are using custom dbPath or logPath files after dropping the existing PSMDB package, the mongod user is dropped as mentioned above. So after the uninstall, and before reinstalling the MongoDB package again, there are some rare cases listed below that can cause permission issues to MongoDB files:

  1. The mongod user is created manually as ordinary user (not using -r option with useradd command which creates system user).
    • drop mongodb packages
    • create mongod user again as ordinary user
    • install mongodb packages
  1. Creating some other system users before installing the MongoDB package. i.e. the newly-created system user might use the same uid/gid of the previously dropped mongod user. This will cause the new mongod user to have different uid/gid while installing the packages.
    • drop mongodb packages
    • create some other system users (which overwrites the pid/gid of dropped mongod user while removing MongoDB package) 
    • install mongodb package

Also, the third scenario is when you have created a mongod user manually as an ordinary user even before installing MongoDB for the first time, which might cause the said issue with the permission for the custom db/log files.

An Example of the Issue

See the example below for the same. The custom dbPath is /data/mongodb:

Install PSMDB 4.0 for the first time and check the directory permission again:

Edit the conf file with custom dbPath and start mongod service:

Now remove the MongoDB Package:

Then check the permission for the data directory and as expected, they look like below – 999:997 which was uid/gid of dropped mongod user/group as part of “yum remove “.

As we have seen in the scenarios above and as per scenario 2, now create a system user/group called testusr as follows and you will be surprised to see the permission of /data/mongodb. This is because the user was created with the same uid/gid of dropped mongod user:

Upgrade to PSMDB 4.2 package now:

Check the permission for the custom data directory path (/data/mongodb) and to the default path(/var/lib/mongo). We notice that the mongod permission for the custom path /data/mongodb was not restored on its own. Also the /etc/mongod.conf is moved to /etc/mongod.conf.rpmsave while installing the PSMDB package, and a new file /etc/mongod.conf is created from the package installation. So you need to edit the conf file as needed or restore the old conf file:

The following error will be received when starting with the lost permission:

Solution:

You have to manually provide mongod user permission and the old setting in config file as before to start with the new version of MongoDB package.

Update the Package Instead of Remove & Install

The other good news is you don’t need to follow these steps if you are updating the package instead of removing and installing the next version. This path doesn’t remove the mongod user and also doesn’t move the /etc/mongod.conf to /etc/mongodb.conf.rpmsave. So you can use this method to upgrade.

While writing this blog post, there were a lot of discussions internally and per them, I have raised this JIRA https://jira.percona.com/browse/PSMDB-546  to update the documentation and reflect this update path instead of removing and installing the newer version.

CONCLUSION

When you are doing upgrade/downgrade, check the permissions of the data directory and the log files before starting the MongoDB service. If needed, provide the permissions of mongod user to the db files to start the service without issue. Hope this helps you if you are facing such issues.

Share this post

Leave a Reply