Navigation

Add Members to a Replica Set

Overview

This tutorial explains how to add an additional member to an existing replica set. For background on replication deployment patterns, see the Replica Set Deployment Architectures document.

Maximum Voting Members

A replica set can have a maximum of seven voting members. To add a member to a replica set that already has seven voting members, you must either add the member as a non-voting member or remove a vote from an existing member.

Init Scripts

In production deployments you can configure a init script to manage member processes.

Existing Members

You can use these procedures to add new members to an existing set. You can also use the same procedure to “re-add” a removed member. If the removed member’s data is still relatively recent, it can recover and catch up easily.

Data Files

If you have a backup or snapshot of an existing member, you can move the data files (e.g. the dbPath directory) to a new system and use them to quickly initiate a new member. The files must be:

  • A valid copy of the data files from a member of the same replica set. See Back Up and Restore with Filesystem Snapshots document for more information.

    Important

    Always use filesystem snapshots to create a copy of a member of the existing replica set. Do not use mongodump and mongorestore to seed a new replica set member.

  • More recent than the oldest operation in the primary’s oplog. The new member must be able to become current by applying operations from the primary’s oplog.

IP Binding

Starting in MongoDB 3.6, MongoDB binaries, mongod and mongos, bind to localhost by default. If the net.ipv6 configuration file setting or the --ipv6 command line option is set for the binary, the binary additionally binds to the localhost IPv6 address.

Previously, starting from MongoDB 2.6, only the binaries from the official MongoDB RPM (Red Hat, CentOS, Fedora Linux, and derivatives) and DEB (Debian, Ubuntu, and derivatives) packages bind to localhost by default.

When bound only to the localhost, these MongoDB 3.6 binaries can only accept connections from clients (including the mongo shell, other members in your deployment for replica sets and sharded clusters) that are running on the same machine. Remote clients cannot connect to the binaries bound only to localhost.

To override and bind to other ip addresses, you can use the net.bindIp configuration file setting or the --bind_ip command-line option to specify a list of hostnames or ip addresses.

Warning

Before binding to a non-localhost (e.g. publicly accessible) IP address, ensure you have secured your cluster from unauthorized access. For a complete list of security recommendations, see Security Checklist. At minimum, consider enabling authentication and hardening network infrastructure.

For example, the following mongod instance binds to both the localhost and the hostname My-Example-Associated-Hostname, which is associated with the ip address 198.51.100.1:

mongod --bind_ip localhost,My-Example-Associated-Hostname

In order to connect to this instance, remote clients must specify the hostname or its associated ip address 198.51.100.1:

mongo --host My-Example-Associated-Hostname

mongo --host 198.51.100.1

Tip

When possible, use a logical DNS hostname instead of an ip address, particularly when configuring replica set members or sharded cluster members. The use of logical DNS hostnames avoids configuration changes due to ip address changes.

Requirements

  1. An active replica set.
  2. A new MongoDB system capable of supporting your data set, accessible by the active replica set through the network.

Otherwise, use the MongoDB installation tutorial and the Deploy a Replica Set tutorials.

Procedures

Prepare the Data Directory

Before adding a new member to an existing replica set, prepare the new member’s data directory using one of the following strategies:

  • Make sure the new member’s data directory does not contain data. The new member will copy the data from an existing member.

    If the new member is in a recovering state, it must exit and become a secondary before MongoDB can copy all data as part of the replication process. This process takes time but does not require administrator intervention.

  • Manually copy the data directory from an existing member. The new member becomes a secondary member and will catch up to the current state of the replica set. Copying the data over may shorten the amount of time for the new member to become current.

    Ensure that you can copy the data directory to the new member and begin replication within the window allowed by the oplog. Otherwise, the new instance will have to perform an initial sync, which completely resynchronizes the data, as described in Resync a Member of a Replica Set.

    Use rs.printReplicationInfo() to check the current state of replica set members with regards to the oplog.

For background on replication deployment patterns, see the Replica Set Deployment Architectures document.

Add a Member to an Existing Replica Set

Tip

When possible, use a logical DNS hostname instead of an ip address, particularly when configuring replica set members or sharded cluster members. The use of logical DNS hostnames avoids configuration changes due to ip address changes.

  1. Start the new mongod instance. Specify the data directory and the replica set name. The following example specifies the /srv/mongodb/db0 data directory and the rs0 replica set:

    mongod --dbpath /srv/mongodb/db0 --replSet rs0  --bind_ip localhost,<hostname(s)|ip address(es)>
    

    Warning

    Before binding to a non-localhost (e.g. publicly accessible) IP address, ensure you have secured your cluster from unauthorized access. For a complete list of security recommendations, see Security Checklist. At minimum, consider enabling authentication and hardening network infrastructure.

    For more information on configuration options, see the mongod manual page.

    Optional

    You can specify the data directory, replica set name, and the ip binding in the mongod.conf configuration file, and start the mongod with the following command:

    mongod --config /etc/mongod.conf
    
  2. Connect to the replica set’s primary.

    You can only add members while connected to the primary. If you do not know which member is the primary, log into any member of the replica set and issue the db.isMaster() command.

  3. Use rs.add() to add the new member to the replica set. Pass the member configuration document to the method. For example, to add a member at host mongodb3.example.net, issue the following command:

    rs.add( { host: "mongodb3.example.net:27017", priority: 0, votes: 0 } )
    

    Tip

    When a newly added secondary has its votes and priority settings greater than zero, during its initial sync, the secondary still counts as a voting member even though it cannot serve reads nor become primary because its data is not yet consistent.

    This can lead to a case where a majority of the voting members are online but no primary can be elected. To avoid such situations, consider adding the new secondary initially with priority :0 and votes :0. Then, once the member has transitioned into SECONDARY state, use rs.reconfig() to update its priority and votes.

  4. Ensure that the new member has reached SECONDARY state. To check the state of the replica set members, run rs.status():

    rs.status()
    
  5. Once the newly added member has transitioned into SECONDARY state, use rs.reconfig() to update the newly added member’s priority and votes if needed.

    For example, if rs.conf() returns the configuration document for mongodb3.example.net:27017 as the fifth element in the members array, to update its priority and votes to 1, use the following sequence of operations:

    var cfg = rs.conf();
    cfg.members[4].priority = 1
    cfg.members[4].votes = 1
    rs.reconfig(cfg)
    

Warning

  • The rs.reconfig() shell method can force the current primary to step down, which causes an election. When the primary steps down, the mongod closes all client connections. While this typically takes 10-20 seconds, try to make these changes during scheduled maintenance periods.
  • Avoid reconfiguring replica sets that contain members of different MongoDB versions as validation rules may differ across MongoDB versions.