Installing and configuring MongoDB on Debian, as well as ReplicaSet and a couple of other little things

This guide describes the step-by-step installation and configuration of a replica of 3 mongoDB nodes based on the WiredTiger engine. As well as some useful little things for people who first encountered MongoDB.

Important clarification:

1. An understanding of the final architecture is required before installation.
2. Some amenities require an Enterprise license.

In my case, this affected the In-Memory Storage Engine, which I wanted to add to test the performance of a database located in RAM.
')

A bit about the WiredTiger engine

Engine WiredTiger is a new mongoDB engine, used by default instead of MMAP, starting from version 3.4. Good for working with data at the level of collections and individual documents, and not completely base. It also fixes the Global lock problem for the above reason, which is why it was chosen in production.

Installing OS and components

We put Debian any convenient version of you, I used 8.6.0.
To simplify the scaling of nodes and databases, I recommend using lvm.
Of the components, you need only ssh for more convenient access to the server.

Install MongoDB.

I use the free Community Edition, the guide will be based on it.
If necessary, I will add the Enterpise version, since she also spun and sorted out.
This manual is described for the 3 server variant (master, slave, arbiter).

So, let's go:

1. First we need to import the public GPG key.
   
   

   sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6 

2. Next, add the MongoDB repository.
   Open the file with the list of repositories.
   
   

    nano /etc/apt/sources.list 

   
   Copy the repository and save the changes.
   
   

    deb http://repo.mongodb.org/apt/debian jessie/mongodb-org/3.4 main 

   
   Update the list of available packages
   
   

    sudo apt-get update 

3. Install MongoDB and dependent packages.
   
   

    sudo apt-get install -y mongodb-org 

Repeat the procedure for 3 (three) servers,
This completes the initial setup.
Now go to the mongoDB setup.

Setting and adding servers to Replica Set

In the beginning was the word
Let's first understand what a replica set is.
Replica Set is a cluster of MongoDB servers that implements the master-slave replication mechanism and automatically switches between them. This is the recommended replication mechanism from MongoDB developers. link to off. documentation.

Picture:



We use the replication mechanism shown in the picture:
master, two secondary and arbitrator.

Now in order:

Primary is the primary mongoDB server.
Secondary - exact copies of the database (s) of data with real-time synchronization.
Arbiter - server selection of the secondary replica with the highest priority, which will be the main in the event of a server crash.

VERY IMPORTANT:
Arbiter is responsible only for the election of a successor, he cannot become a successor himself, therefore it is recommended to give minimum resources to an arbitrator.

MORE IMPORTANT:
Technically, you can live without an arbitrator at all, but with him the elections will occur many times faster, respectively, the downtime will be minimized. Plus, there is a non-zero chance of losing ReplicaSet entirely.

Primary
We write the settings in the /etc/mongod.conf file. My file looks like this:

 storage: engine: wiredTiger dbPath: /var/lib/mongodb journal: enabled: true systemLog: destination: file logAppend: true path: /var/log/mongodb/mongod.log replSetName: replicaname net: port: 27017 bindIp:0.0.0.0 

Specified variable values:
dbPath - the path to the database. When pointing to an empty space, it will create bases there. When deploying, the backup will pick up the existing one.
journal - includes journaling.
replSetName is the name of the replica. Must be identical on all nodes within the replica.
bindIp - a list of addresses from which you can receive connections on port 27017.

Next, for configuration, I used mongoDB itself:

 mongo 

We get inside and first check the status:

 rs.status() 

Initial status must be 0 - startup. Indicates that the node is not a member of any replicas.

Initialize the replica.
Run in MongoShell on the primary node

 rs.initiate( { _id: "rs0", version: 1 members: [ { _id : 0, host : "<Primary server ip>:27017" } ] }) 

Check the configuration:

 rs.conf() 

Immediately after adding a node to the replica, it will be in status 5 - Startup2, this means that it has joined the replica and is currently synchronizing. May take a long time.

Add the following nodes:

 rs.add("<Secondary server ip>:27017") 

Add an arbitrator:

 rs.addArb(“<arbiter server ip>:27017”) 

or

 rs.add(“<arbiter server ip>:27017”, true) 

The statuses in rs.status () will be:
1 - Primary
2 - Secondary
7 - Arbiter

Priorities

Execution options

The first:
It is necessary to put priorities (the numbers of member id are taken from the status)

 cfg = rs.conf() cfg.members[0].priority = 2 cfg.members[1].priority = 3 cfg.members[2].priority = 1 rs.reconfig(cfg, {force : true}) 

The higher the priority number, the lower the priority itself when choosing a Primary node.

Second:
Add nodes with a pre-assigned priority:

 rs.add({host: “<secondary server ip:27017”, priority: 1}) 

We check that all nodes are available and set up with the right priorities:

 rs.status() 

In my case, configs and statuses acquired the following form:

 rs.status() 

 { "set" : "<  >", "date" : ISODate("2017-05-05T13:57:01.538Z"), "myState" : 2, "term" : NumberLong(1021), "syncingTo" : "<secondary server ip>:27017", "heartbeatIntervalMillis" : NumberLong(2000), "members" : [ { "_id" : 0, "name" : "<Secondary server ip>:27017", "health" : 1, "state" : 2, "stateStr" : "SECONDARY", "uptime" : 1908285, "optime" : { "ts" : Timestamp(1493992502, 1), "t" : NumberLong(1021) }, "optimeDate" : ISODate("2017-05-05T13:55:02Z"), "syncingTo" : ":27017", "configVersion" : 373058, "self" : true }, { "_id" : 1, "name" : "<Primary server ip>:27017", "health" : 1, "state" : 1, "stateStr" : "PRIMARY", "uptime" : 1688032, "optime" : { "ts" : Timestamp(1493992502, 1), "t" : NumberLong(1021) }, "optimeDate" : ISODate("2017-05-05T13:55:02Z"), "lastHeartbeat" : ISODate("2017-05-05T13:57:00.108Z"), "lastHeartbeatRecv" : ISODate("2017-05-05T13:57:00.895Z"), "pingMs" : NumberLong(0), "electionTime" : Timestamp(1492304555, 1), "electionDate" : ISODate("2017-04-16T01:02:35Z"), "configVersion" : 373058 }, { "_id" : 2, "name" : "<Arbiter server ip>:27017", "health" : 1, "state" : 7, "stateStr" : "ARBITER", "uptime" : 1680982, "lastHeartbeat" : ISODate("2017-05-05T13:56:59.812Z"), "lastHeartbeatRecv" : ISODate("2017-05-05T13:56:59.062Z"), "pingMs" : NumberLong(0), "configVersion" : 373058 } ], "ok" : 1 } 

When you add a node to an existing replica, it will be unavailable for some time due to synchronization.

The installation and configuration of MongoDB in ReplicaSet mode is complete and you can fill the database with a clear conscience.

Other useful commands

Here, in general, the whole story about my experience of deploying a system unseen before in production. I would be glad to criticism and additions, if any.