Downloads

Blog

Contact Us

Installing MongoDB 3.2 and upgrading MongoDB replica set

June 15, 2016

Author

David Murphy

MongoDB

Share this Post:

In this post, we’ll examine a couple of ways for upgrading MongoDB replica set.

With the release of MongoDB 3.2, comes a rash of new features and improvements. One of these enhancements is improved replica sets. From MongoDB: “A replica set in MongoDB is a group of mongod processes that maintain the same data set. Replica sets provide redundancy and high availability, and are the basis for all production deployments.”

Config servers are replica sets!

This is HUGE. It signals a significant advancement in backups, metadata stability and overall maturity. It is a very long-awaited feature that shows MongoDB is maturing. It means:

Mongos’ can retry connection vs error

Unified and consistent backups!

Up to 50 secondaries
- Remove bugs with Mongos’ not near config servers!

How do we activate all these new awesome features? Let’s do it!

Upgrading to 3.2

Replace binaries and restart one secondary at a time

Then primaries as well

Restart configs in reverse order
- If configdb=con1, con2, con3
  - Restart con3, con2, and then finally con1 with 3.2
  - Do con1 as FAST as possible, while the balancer is also disabled
- You no longer need to restart a mongos –upgrade (as of 3.2)
- Restart all mongos, this will reset ALL connections at some point (whether you do at once or space it out).

Upgrading the replset to the new protocol

This is by far the easiest upgrade bit but DON’T do it until you know your stable on 3.2. Log into each primary and run:

>cfg = rs.conf();<br> {<br>     "_id" : "r1",<br>     "version" : 2,<br>     "members" : [<br>         {<br>             "_id" : 0,<br>             "host" : "localhost:17001"<br>         },<br>         {<br>             "_id" : 1,<br>             "host" : "localhost:17002"<br>         },<br>         {<br>            "_id" : 2,<br>            "host" : "localhost:17003",<br>         }<br>     ]<br>}<br>>cfg.protocolVersion=1;<br>>rs.reconfig(cfg);<br>{<br>    "ok" : 1,<br>    "$gleStats" : {<br>        "lastOpTime" : Timestamp(1464947003, 1),<br>        "electionId" : ObjectId("7fffffff0000000000000018")<br>    }<br>}

1

>cfg = rs.conf(); { "_id" : "r1", "version" : 2, "members" : [ { "_id" : 0, "host" : "localhost:17001" }, { "_id" : 1, "host" : "localhost:17002" }, { "_id" : 2, "host" : "localhost:17003", } ] } >cfg.protocolVersion=1; >rs.reconfig(cfg); { "ok" : 1, "$gleStats" : { "lastOpTime" : Timestamp(1464947003, 1), "electionId" : ObjectId("7fffffff0000000000000018") } }

Or:

>db.getSiblingDB('config').shards.forEach(function(shard){<br>    x = new Mongo(shard.host); /* Assumes no auth needed */<br>    conf =x.getDB("local").system.replset.findOne()<br>    conf.protcolVersion=1;conf.version++;<br>    x.getDB('admin').runCommand({ replSetReconfig: conf });<br>});

1	>db.getSiblingDB('config').shards.forEach(function(shard){<br> x = new Mongo(shard.host); /* Assumes no auth needed */<br> conf =x.getDB("local").system.replset.findOne()<br> conf.protcolVersion=1;conf.version++;<br> x.getDB('admin').runCommand({ replSetReconfig: conf });<br>});

The quick upgrade scripts

Upgrade_all_to_3.2.4.sh

Does what it says: kills every process and launches them on 3.2 binaries with no other changes.

Upgrade_replica_proto_version.sh

Simply runs the quick rs.reconfig() on each primary, adds the new settings to enable to new replication features.

Let’s upgrade the configs the right way!

This is not included as part of a normal upgrade so only do this AFTER you’re stable and don’t do it before upgrading the protocolVersion we just talked about. (I mean it! Disregard this advice and your life will seriously not be awesome!)

Upgrading to a Config ReplicaSet ( the official way)

Run rs.initiate on the first config in the list (must be 3.2.4+)
- Must be a fully configured document with configsrv:true defined.

Restart same config server adding
- configsvrMode = sccc
- replSet = <name used in rs.initiate()>
- storageEngine= WiredTiger

Start the new config servers for the other two nodes (should be a new dbpath and port)

Add those nodes to the replSet and check their status

Remove the second original config server from the running

Restart the 1st node you set “sccc” on to not have that setting

At this point, the 1st node will transition to removed if using MMAP.

Restart a mongos with a new configdb line
- –configdb <replSetName>/node1:port,node2:port,…
- Only replset members should be listed

Verify you can work and query through mongos

Repeat on all mongos

Remove the 1st node with rs.remove

Shutdown final original config and enable balancer

There is also an easy way.

The easy way, with a small maintenance window, which lets you just restore a good backup and have a nice and simple rollback plan:

Stop all mongos after backing up the config directory

Run rs.initiate on first config server

Stop the 2nd, then the 3^rd, restarting them with an empty dbpath directory

Check the rs.status now

Stop the 1st config server and restart with an empty dbpath directory

Check Status

Restart all mongos, adding <replSetName>/ to the front of the configdb line.

Done!

Oh look there is a quick script we have made for you:

Upgrade_config_to_repliaset.sh
- Kill all config and mongos processes
- Restart the first config server on non-standard port
- Mongodump config database
- Restart c1 as WiredTiger, clearing that data path
- Import dump back into first config server
- Restart on normal port
- Initialize Replica Set
- Restart second and third config server after clearing dbpath folder
- After the initial sync, start all the mongos.
- Done and script exits!