Reference links:
MongoDB is made ofC++written in language，It is an open source database system based on distributed file storage.。 under high load，Add more nodes，Can guarantee server performance。 MongoDB intended forWEBApplications provide scalable, high-performance data storage solutions。 MongoDB Store data as a document，The data structure consists of key values(key=>value)pair composition。MongoDB The documentation is similar to JSON object。Field values can contain other documents，Arrays and document arrays。
Main features MongoDB Is a database for document storage，Relatively simple and easy to operate。 you canMongoDBSets the index of any attribute in the record (like：FirstName="Sameer",Address="8 Gandhi Road")to achieve faster sorting。 You can create data mirrors locally or over the network，This makesMongoDBHave stronger scalability。 like果负载的增加（Need more storage space and more processing power） ，It can be distributed across other nodes in a computer network. This is called sharding.。 MongoSupport rich query expressions。Query command usageJSONform mark，Easily query objects and arrays embedded in documents。 MongoDb useupdate()The command can replace the completed document（data）或者一些指定的data字段 。 MongodbinMap/reduce主要yes用来对data进行批量处理and聚合operate。 MapandReduce。Mapfunction callemit(key,value)Iterate through all records in the collection，Willkeyandvaluepass toReducefunction to process。 Map函数andReduce函数yesuseJavascriptwritten，and can be passed ormapreducecommand to executeMapReduceoperate。 GridFSyesMongoDBin一个内置功能，Can be used to store a large number of small files。 MongoDBAllow script execution on the server side，availableJavascriptwrite a function，Executed directly on the server side，You can also store the function definition on the server side，Just call it directly next time。 MongoDBSupport various programming languages:RUBY，PYTHON，JAVA，C++，PHP，C# and many other languages.MongoDBEasy to install。
#1.Install dependency packagessudo yum install libcurl openssl #centos redhat
sudo apt-get install libcurl4 openssl ubantu 18.04 LTS
#2. Download the source code package Reference link:
MongoDB Source code download address：/download-center#community
#3. Download binary packagewget /linux/mongodb-linux-x86_64-rhel70-4.2. # download
#4. Unzip to a specific directorymv mongodb-linux-x86_64-rhel70-4.2. /usr/local/mongodb4 #Copy the decompressed package to the specified directory
#5. Configure environment variablesecho "export PATH=$PATH:/usr/local/mongodb4/bin" >>/etc/profile
source /etc/profile
#6.Create relevant directoriesData storage directory：/var/lib/mongodb
Log file directory：/var/log/mongodb
sudo mkdir -p /var/lib/mongo
sudo mkdir -p /var/log/mongodb
sudo chown `whoami` /var/lib/mongo # Set permissionssudo chown `whoami` /var/log/mongodb # Set permissions
#7. Start the mongodb service. The default port is: 10503 mongod --dbpath /var/lib/mongo --logpath /var/log/mongodb/ --fork
[root@shell ~23:18:56]# mongod --dbpath /var/lib/mongo --logpath /var/log/mongodb/ --fork
about to fork child process, waiting until server is ready for connections.
forked process: 10503
child process started successfully, parent exiting
[root@shell ~23:19:10]# netstat -lntup |grep mongo
tcp 0 0 127.0.0.1:27017 0.0.0.0:* LISTEN 10503/mongod
#Parameter description:-h:Specifies the database hostIP
-u:Specifies the user name of the database
-p:Specify the password for the database
-d:Specify the name of the database
-c:specifycollectionname
-o:specify到要导出的文件名
-q:specify导出的过滤条件
-j,--numParallclCollections= number of collectiions to dum in parallel
#8. Stop mongodbMethod one:
mongod --dbpath /var/lib/mongo --logpath /var/log/mongodb/ --shutdown
Method two:
> use admin
switched to db admin
> ()
#9. Check the log file to see if the startup is successful[root@shell ~23:19:25]# tail -10f /var/log/mongodb/
2021-10-16T23:19:10.399+0800 I STORAGE [LogicalSessionCacheRefresh] createCollection: with provided UUID: 520fda48-31c3-45f9-8394-048c2b664b49 and options: { uuid: UUID("520fda48-31c3-45f9-8394-048c2b664b49") }
2021-10-16T23:19:10.400+0800 I NETWORK [listener] Listening on /tmp/
2021-10-16T23:19:10.401+0800 I NETWORK [listener] Listening on 127.0.0.1
2021-10-16T23:19:10.401+0800 I NETWORK [listener] waiting for connections on port 27017
2021-10-16T23:19:10.414+0800 I INDEX [LogicalSessionCacheRefresh] index build: done building index _id_ on ns
2021-10-16T23:19:10.424+0800 I INDEX [LogicalSessionCacheRefresh] index build: starting on properties: { v: 2, key: { lastUse: 1 }, name: "lsidTTLIndex", ns: "", expireAfterSeconds: 1800 } using method: Hybrid
2021-10-16T23:19:10.425+0800 I INDEX [LogicalSessionCacheRefresh] build may temporarily use up to 200 megabytes of RAM
2021-10-16T23:19:10.425+0800 I INDEX [LogicalSessionCacheRefresh] index build: collection scan done. scanned 0 total records in 0 seconds
2021-10-16T23:19:10.426+0800 I INDEX [LogicalSessionCacheRefresh] index build: inserted 0 keys from external sorter into index in 0 seconds
2021-10-16T23:19:10.427+0800 I INDEX [LogicalSessionCacheRefresh] index build: done building index lsidTTLIndex on ns
#1. Enter the bin directory of the installation path cd /usr/local/mongodb4/bin
enter
./mongo +Enter
[root@shell /usr/local/mongodb4/bin23:22:58]# mongo
MongoDB shell version v4.2.17
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("9370b83b-492c-4f62-b54d-e7cdadce6f06") }
MongoDB server version: 4.2.17
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
/
Questions? Try the MongoDB Developer Community Forums
Server has startup warnings:
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten]
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten] ** WARNING: Access control is not enabled for the database.
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten] ** Read and write access to data and configuration is unrestricted.
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten]
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten] ** WARNING: This server is bound to localhost.
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten] ** Remote systems will be unable to connect to this server.
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten] ** Start the server with --bind_ip <address> to specify which IP
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten] ** addresses it should serve responses from, or with --bind_ip_all to
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten] ** bind to all interfaces. If this behavior is desired, start the
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten] ** server with --bind_ip 127.0.0.1 to disable this warning.
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten]
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten]
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten]
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten] ** We suggest setting it to 'never'
2021-10-16T23:19:10.328+0800 I CONTROL [initandlisten]
---
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).
The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.
To enable free monitoring, run the following command: ()
To permanently disable this reminder, run the following command: ()
#2.Test data> 1 +1
2
> 1+4
5
> 100*100
10000
#Insert data> ({x:10})
WriteResult({ "nInserted" : 1 })
#View data> ()
{ "_id" : ObjectId("616aee731b401b9fc72193da"), "x" : 10 }
#1. Full database backupmkdir /mongodb/backup mongodump -uroot -proot123 --port 27017 --authenticationDatabase admin -o /mongodb/backup #2. Back up the world librarymongodum -uroot -proot123 --port 27017 --authenticationDatabase admin -d world -o /mongodb/backup #3. Back up the Log collection under the oldboy librarymongodum -uroot -proot123 --port 27017 --authenticationDatabase admin -d oldboy -c log -o /mongodb/backup #4. Compress backupmongodum -uroot -proot123 --port 27017 --authenticationDatabase admin -o /mongodb/backup/ --gzip #5.Restore the test librarymongorestore -uroot -proot123 --port 27017 --authenticationDatabase admin -d test1 /mongodb/backup/test --gzip
1. Mongod basic operation and maintenance part
1.Install MongoDB
#Software download
/try/download/community
#2. Upload and decompress mongod software
mkdir -p /mongodb && cd /mongodb
rz mongodb-linux-x86_64-rhel70-4.2.
tar xf mongodb-linux-x86_64-rhel70-4.2.
mv mongodb-linux-x86_64-rhel70-4.2. /usr/local/mongodb
#3. Add environment variables
vim /etc/profile
export PATH=/usr/local/mongodb/bin:$PATH
source /etc/profile
#4. Turn off THP
Under root user
Add the following code at the end of vim /etc/
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
Check:
[root@lss mongodb]# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never
[root@lss mongodb]# cat /sys/kernel/mm/transparent_hugepage/defrag
[always] madvise never
For other system shutdowns, please refer to the official documentation:
/manual/tutorial/transparent-huge-pages/
Why close?
Transparent Huge Pages (THP) is a Linux memory management system
that reduces the overhead of Translation Lookaside Buffer (TLB)
lookups on machines with large amounts of memory by using larger memory pages.
However, database workloads often perform poorly with THP,
because they tend to have sparse rather than contiguous memory access patterns.
You should disable THP on Linux machines to ensure best performance with
MongoDB.
Transparent Huge Pages (THP) is a Linux memory management system that reduces the translation lookup buffer (TLB) overhead by using larger memory page lookups on machines with large amounts of memory. However, database workloads often perform poorly when using THP because they tend to have sparse rather than contiguous memory access patterns. You should disable THP on your Linux machine to ensure optimal performance of MongoDB.
2.Environmental preparation
(1)Create the required users and groups
useradd mongod
passwd mongod
entermongod（password）
(2)createmongodbRequired directory structure
mkdir -p /mongodb/conf
mkdir -p /mongodb/log
mkdir -p /mongodb/data
(3) Modify permissions
chown -R mongod:mongod /mongodb
(4) Switch users and set environment variables
su - mongod
vi .bash_profile
export PATH=/usr/local/mongodb/bin:$PATH
source .bash_profile
3. Start the database and initialize the data
#1.Method one：Direct command to start
su - mongod
mongod --dbpath=/mongodb/data --logpath=/mongodb/log/ --port=27017 --logappend --fork
[mongod@lss conf]$ mongo
MongoDB shell version v4.2.8
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("f52da758-bd97-4c95-a54c-d6276d8c7dc1") }
MongoDB server version: 4.2.8
show databases;
use admin;
show tables;
#2. Method 2: Start with configuration file
1. General configuration documents
vim /mongodb/conf/
logpath=/mongodb/log/
dbpath=/mongodb/data
port=27017
logappend=true
fork=true
# Close mongodb
mongod -f /mongodb/conf/ --shutdown
Start mongodb using configuration file
mongod -f /mongodb/conf/
#Login to mongo requires switching to use admin
() You can also close mongod
Format configuration document
# YAML configuration file application
--
NOTE：
YAML does not support tab characters for indentation: use spaces instead.
--System log related
systemLog:
destination: file
path: "/mongodb/log/" --Log location
logAppend: true --The log is recorded in append mode
--Data storage related
storage:
journal:
enabled: true
dbPath: "/mongodb/data" --The location of the data path
-- Process control
processManagement:
fork: true --Background daemon process
pidFilePath: <string> --The location of the pid file generally does not need to be configured. You can remove this line and it will automatically
Generate into data
--Network configuration related
net:
bindIp: <ip> -- Listening address, if this line is not configured, the listening address is 0.0.0.0
port: <port> --Port number, the port number is not configured by default, it is 27017
-- Configuration related to security verification
security:
authorization: enabled --Whether to turn on username and password verification
------------------The following are replica sets related to sharded clusters----------------------
replication:
oplogSizeMB: <NUM>
replSetName: "<REPSETNAME>"
secondaryIndexPrefetch: "all"
sharding:
clusterRole: <string>
archiveMovedChunks: <boolean>
---for mongos only
replication:
localPingThresholdMs: <int>
sharding:
configDB: <string>
---
.........
++++++++++++++++++++++
#Write a simple configuration file
YAML example
vim /mongodb/conf/
systemLog:
destination: file
path: "/mongodb/log/"
logAppend: true
storage:
journal:
enabled: true
dbPath: "/mongodb/data/"
processManagement:
fork: true
net:
port: 27017
bindIp: 10.0.0.150,127.0.0.1
#closure:
mongod -f /mongodb/conf/ --shutdown
#open
mongod -f /mongodb/conf/
[mongod@lss conf]$ mongod -f /mongodb/conf/
about to fork child process, waiting until server is ready for connections.
forked process: 8703
child process started successfully, parent exiting
++++++++++++++++++++++
#mongodb shutdown method
mongod -f --shutdown
#Loginmongo
() You can also close mongod
4. Basic user management
1. Precautions
User management *****
Notice：
Validation library，When creating a useruseReached library，when using user，要加上Validation library才能登陆。
For admin users,must be inadminCreated under.
1. When creating a user, the library used is the verification library for this user.
2. When logging in, you must explicitly specify the verification library to log in.
3. Usually, the verification library used by administrators is admin, and the verification library used by ordinary users is usually the managed library set as the verification library.
4. If you log in to the database directly without using use, the default verification library is test, which is not recommended by our production.
2. Basic grammar
3. User instance
（1）--Create super administrator：Manage all databases（mustuse adminCreate again）
$ mongo
> use admin
switched to db admin
> (
... {
... user: "root",
... pwd: "root123",
... roles: [ { role: "root", db: "admin" } ]
... }
... )
Successfully added user: {
"user" : "root",
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}
#Verify user
> ('root','root123')
1
(2).Create a normal user
(
{
user: "app01",
pwd: "app01",
roles: [ { role: "readWrite" , db: "app" } ]
}
)
> ('app01','app01')
1
(3). Turn on authentication, write the following information into the configuration file, and restart mongodb
In the configuration file, add the following configuration
security:
authorization: enabled
Restart mongodb
mongod -f /mongodb/conf/ --shutdown
mongod -f /mongodb/conf/
(4). Log in to mongo again
[root@lss ~]# mongo -u root -p root123 10.0.0.150/admin
MongoDB shell version v4.2.8
connecting to: mongodb://10.0.0.150:27017/admin?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("13dd260c-8b0f-4e6c-b573-534cdbdc8f93") }
MongoDB server version: 4.2.8
#View users:
use admin #To check the user, you need to check what the user is using
().pretty()
> ().pretty()
{
"_id" : "",
"userId" : UUID("32c5bce8-6bbb-4e4d-be46-35f3d0adcdb6"),
"user" : "root",
"db" : "admin",
"credentials" : {
"SCRAM-SHA-1" : {
"iterationCount" : 10000,
"salt" : "UvY7mHKBVOb6jFDWTvYrIg==",
"storedKey" : "QNaK1pyuQ4wFmjQf91dABFJgWuU=",
"serverKey" : "/iEKzPvH6gb43ZIFeAypcSoUlJo="
},
"SCRAM-SHA-256" : {
"iterationCount" : 15000,
"salt" : "5rclUKutvHdivPkkV5GNhq0txt+PRL3FiqSetQ==",
"storedKey" : "EJJDiRMzEUd0BH1R4uBfaA9QWB4cV6j6fuz+M9L50fI=",
"serverKey" : "uJTtn0d2/L3YtzJjSRC9X/L48AkC+ADZ6yFsZhdpgZM="
}
},
"roles" : [
{
"role" : "root",
"db" : "admin"
}
]
}
{
"_id" : "admin.app01",
"userId" : UUID("3171b30e-1864-4217-a575-41325818535a"),
"user" : "app01",
"db" : "admin",
"credentials" : {
"SCRAM-SHA-1" : {
"iterationCount" : 10000,
"salt" : "4Z6hmuTdAqcpPURUP+Z3Tw==",
"storedKey" : "2laFEA3Jk2Z8GVwUIJUBgSv2bGw=",
"serverKey" : "neMzwFjlw6BkbskSKnHBfU/bO+g="
},
"SCRAM-SHA-256" : {
"iterationCount" : 15000,
"salt" : "rPIdl7z5nUX02p8xXfRrbOuqYLpFSOWz08kg/w==",
"storedKey" : "8P9fn0GONuGXi3cGbu5nDV/AX8vBHYFOaJ2NjR+WR3o=",
"serverKey" : "wsoiuAdTB63UnJB3/c1Ya+B8Yeo5m9vlKsC5gr3Dfoc="
}
},
"roles" : [
{
"role" : "readWrite",
"db" : "app"
}
]
}
(5). Create user app01 with read and write permissions on the app database:
1. Log in as super administrator user
mongo -uroot -proot123 admin
2. Choose a verification library
use app #If the app library does not exist, you can also use the app library.
3.Create user
(
{
user: "app01",
pwd: "app01",
roles: [ { role: "readWrite" , db: "app" } ]
}
)
mongo -uapp01 -papp01 app
4. Create a user with read and write permissions for the app database and read permission for the test database:
mongo -uroot -proot123 10.0.0.130/admin
use app
(
{
user: "app03",
pwd: "app03",
roles: [ { role: "readWrite", db: "app" },
{ role: "read", db: "test" }
]
}
)
5. Query user information in mongodb
mongo -uroot -proot123 10.0.0.150/admin
show databases;
use amdin;
show tables;
().pretty()
> show databases;
admin 0.000GB
config 0.000GB
local 0.000GB
> use admin;
switched to db admin
> show tables;
6. Delete the user (log in as root, use to the verification library)
# mongo -uroot -proot123 10.0.0.130/admin
use app
("admin")
4. Role relationship
Basic CRUD
1. General methods and help obtaining
#a.Get help
help
> help
() help on db methods
() help on collection methods
() sharding helpers
() replica set helpers
();
DB methods:
(nameOrDocument) - switches to 'admin' db, and runs command
[just calls (...)]
([pipeline], {options}) - performs a collectionless aggregation
on this database; returns a cursor
(username, password)
(fromhost) - will only function with MongoDB 4.0 and below
...
> db.();
DBCollection help
db.().help() - show DBCursor help
db.( operations, <optional params> ) - bulk execute write
operations, optional parameters are: w, wtimeout, j
db.( query = {}, <optional params> ) - count the number of documents
that matches the query, optional parameters are: limit, skip, hint, maxTimeMS
...
db.[TAB][TAB]
db.t1.[TAB][TAB]
#Example:
> ()
DB methods:
(nameOrDocument) - switches to 'admin' db, and runs command [just calls (...)]
([pipeline], {options}) - performs a collectionless aggregation on this database; returns a cursor
(username, password)
(fromhost) - will only function with MongoDB 4.0 and below
(name) returns the help for the command
(fromdb, todb, fromhost) - will only function with MongoDB 4.0 and below
(name, {size: ..., capped: ..., max: ...})
(userDocument)
(name, viewOn, [{$operator: {...}}, ...], {viewOptions})
() displays currently executing operations in the db
(writeConcern)
(username)
() - deprecated
() flush data to disk and lock server for backups
() unlocks server following a ()
(cname) same as db['cname'] or
([filter]) - returns a list that contains the names and options of the db's collections
()
() - just returns the err msg string
() - return full status object
()
() get the server connection object
().setSlaveOk() allow queries on a replication slave server
()
() - deprecated
() - returns if profiling is on and slow threshold
()
(name) get the db at the same server as this one
() - returns the write concern used for any operations on this db, inherited from server object if set
() get details about the server's host
() check replica primary status
(opid) kills the current operation in the db
() lists all the db commands
() loads all the scripts in
()
()
()
()
()
()
(cmdObj) run a database command. if cmdObj is a string, turns it into {cmdObj: 1}
()
(level,<component>)
(level,slowms) 0=off 1=slow 2=all
(flag) display extra information in shell output
(<write concern doc>) - sets the write concern for writes to the db
()
()
(<write concern doc>) - unsets the write concern for writes to the db
() current version of the server
() - opens a change stream cursor for a database to report on all changes to its non-system collections.
> ()
() { replSetGetStatus : 1 } checks repl set status
() { replSetInitiate : null } initiates set with default settings
(cfg) { replSetInitiate : cfg } initiates set with configuration cfg
() get the current configuration object from
(cfg) updates the configuration of a running replica set with cfg (disconnects)
(hostportstr) add a new member to the set with default attributes (disconnects)
(membercfgobj) add a new member to the set with extra attributes (disconnects)
(hostportstr) add a new member which is arbiterOnly:true (disconnects)
([stepdownSecs, catchUpSecs]) step down as primary (disconnects)
(hostportstr) make a secondary sync from the given member
(secs) make a node ineligible to become primary for the time specified
(hostportstr) remove a host from the replica set (disconnects)
() allow queries on secondary nodes
() check oplog size and time range
() check replica set members and replication lag
() check who is primary
reconfiguration helpers disconnect from the database so the shell will display
an error, even if the command succeeds.
#b. Common operations
//View current db version
test> ()
//Display the current database
test> db
> ()
//Query all databases
test> show dbs
//Switch database
> use local
//Show current database status
View local data
test> use local
local> ()
View the connection machine address of the current database
> ()
// Specify the database to connect: (default connects to the local test database)
# mongo 192.168.1.24/admin
[mongod@mongodb ~]$ mongo 192.168.1.24/admin
#c. Library and table operations
//Build database
use test
// delete
> ()
{ "dropped" : "test", "ok" : 1 }
//Create a collection (table)
Method 1:
admin> use app #Direct usb app means switching to the app library. If there is no such library, create it directly. This library can be found only when there are tables and data in the library.
app> ('a')
app> ('b')
Method 2: When a document is inserted, a collection is automatically created.
admin> use app
switched to db app
app> ({username:"mongodb"}) #Input data into the table C. If the table does not exist, create it directly. If there is data in the table, you can find the table.
WriteResult({ "nInserted" : 1 })
app> show collections
app> ()
{ "_id" : ObjectId("5743c9a9bf72d9f7b524713d"), "username" : "mongodb" }
// Delete collection
app> use app
switched to db app
app> () //Delete collection
//Rename collection
app> ("log1")
2. Use insert to complete the insertion operation
Operation format：
db.<gather>.insertOne(<JSONobject>) #db.Table name.insert({json object}).insertMany([ 1>, <JSON 2>, …<JSON n>])
Example:
({name: "apple"})
([
{name: "apple"},
{name: "pear"},
{name: "orange"}
])
Insert data in batches:
for(i=0;i<10000;i++){ ({"uid":i,"name":"mongodb","age":6,"date":new
Date()}); }
#View the current table
> show tables;
fruit
> show collections;
fruit
#View the contents of the table
> ()
{ "_id" : ObjectId("62357cef2382a7643d8eea8e"), "name" : "apple" }
3. Use find to query documents
# about find:
find is the basic instruction for querying data in MongoDB, equivalent to SELECT in SQL.
find returns a cursor.
# find Example:
( { "year" : 1975 } ) //Single condition query
( { "year" : 1989, "title" : "Batman" } ) //Multiple conditions and query
( { $and : [ {"title" : "Batman"}, { "category" : "action" }] } )
// Another form of and
( { $or: [{"year" : 1989}, {"title" : "Batman"}] } ) //Multiple conditions or query
( { "title" : /^B/} ) //Search by regular expression
4. Query condition comparison table
| SQL | MQL |
|---|---|
| a=1 | {a: 1} |
| a<>1 | {a: {$ne: 1}} |
| a>1 | {a: {$gt: 1}} |
| a>=1 | {a: {$gte: 1}} |
| a<1 | {a: {$lt: 1}} |
| a <= 1 | {a: {$lte: 1}} |
5. Query logical comparison table
| SQL | MQL |
|---|---|
| a = 1 AND b = 1 | {a: 1, b: 1} or {$and: [{a: 1}, {b: 1}]} |
| a = 1 OR b = 1 | {$or: [{a: 1}, {b: 1}]} |
| a IS NULL | {a: {$exists: false}} |
| a IN (1, 2, 3) | {a: {$in: [1, 2, 3]}} |
6. Query logical operators
● $lt: exists and is less than
● $lte: exists and is less than or equal to
● $gt: exists and is greater than
● $gte: exists and is greater than or equal to
● $ne: does not exist or exists but is not equal to
● $in: exists and is in the specified array
● $nin: does not exist or is not in the specified array
● $or: Match one of two or more conditions
● $and: matches all conditions
7. Use find to search subdocuments
find Support use“field.sub_field”Query subdocuments in the form。Suppose there is a document：
({
name: "apple",
from: {
country: "China",
province: "Guangdong" }
})
Correct way to write:
> ({ "" : "China" })
{ "_id" : ObjectId("6235ab212382a7643d8eea8f"), "name" : "apple", "from" : { "country" : "China", "province" : "Guangdong" } }
> ().pretty()
{ "_id" : ObjectId("62357cef2382a7643d8eea8e"), "name" : "apple" }
{
"_id" : ObjectId("6235ab212382a7643d8eea8f"),
"name" : "apple",
"from" : {
"country" : "China",
"province" : "Guangdong"
}
}
> ({ "" : "Guangdong" })
{ "_id" : ObjectId("6235ab212382a7643d8eea8f"), "name" : "apple", "from" : { "country" : "China", "province" : "Guangdong" } }
8. Search arrays using find
find Supports searching for elements in arrays。Suppose there is a document：
([
{ "name" : "Apple", color: ["red", "green" ] },
{ "name" : "Mango", color: ["yellow", "green"] }
])
View a single condition:
({color: "red"})
Query multiple conditions:
({$or: [{color: "red"}, {color: "yellow"}]} )
9. Use find to search for objects in an array
#Consider the following document，Search within
({
"title" : "Raiders of the Lost Ark",
"filming_locations" : [
{ "city" : "Los Angeles", "state" : "CA", "country" : "USA" },
{ "city" : "Rome", "state" : "Lazio", "country" : "Italy" },
{ "city" : "Florence", "state" : "SC", "country" : "USA" }
]
})
// Find records whose city is Rome
({"filming_locations.city": "Rome"})
When searching for multiple fields of sub-objects in an array, if you use$elemMatch, which means it must be the same
The child object satisfies multiple conditions.
Consider the following two queries:
('movies').find({
"filming_locations.city": "Rome",
"filming_locations.country": "USA"
})
( {
"title" : "11111",
"filming_locations" : [
{ "city" : "bj", "state" : "CA", "country" : "CHN" },
{ "city" : "Rome", "state" : "Lazio", "country" : "Italy" },
{ "city" : "tlp", "state" : "SC", "country" : "USA" }
]
})
('movies').find({
"filming_locations": {
$elemMatch:{"city":"bj", "country": "CHN"}
}
})
10. Control the fields returned by find
find You can specify to return only specified fields；
● _idFields must explicitly indicate not to be returned，Otherwise return by default；
● exist MongoDB In we call this projection（projection）；
● ({},{"_id":0, title:1})
11. Use remove to delete documents
remove The command needs to be used in conjunction with the query conditions；
● Documents matching the query criteria will be deleted；
● Specifying an empty document condition will delete all documents；
● Example below：
( { a : 1 } ) // Delete records with a equal to 1
( { a : { $lt : 5 } } ) // Delete records with a less than 5
( { } ) // Delete all records
() //Report error
12.Use update to update the document
Update Operation execution format：db.<gather>.update(<Query conditions>, <Update fields>)
● Take the following data as an example：
([
{name: "apple"},
{name: "pear"},
{name: "orange"}
])
({name: "apple"}, {$set: {from: "China"}})
● Using updateOne means that no matter how many records match the condition, only the first one will be updated;
● Use updateMany to update as many items as the conditions match;
● The updateOne/updateMany method requires that the update condition part must have one of the following, otherwise an error will be reported:
• $set/$unset
• $push/$pushAll/$pop
• $pull/$pullAll
• $addToSet
● // Report error
({name: "apple"}, {from: "China"})
13. Use update to update the array
● $push: Add an object to the bottom of the array
● $pushAll: Add multiple objects to the bottom of the array
● $pop: Remove an object from the bottom of the array
● $pull: If the specified value is matched, remove the corresponding object from the array
● $pullAll: If any value is matched, delete the corresponding object from the data
● $addToSet: Add a value to the array if it does not exist
14. Use drop to delete a collection
● use db.<gather>.drop() 来删除一个gather
● All documents in the collection will be deleted
● Collection-related indexes will also be deleted
()
15. Use dropDatabase to delete the database
● use () to delete the database
● The corresponding files in the database will also be deleted.，Disk space will be freed
use tempDB
()
show collections // No collections
show dbs // The db is gone
6. Operate mongodb through python
1. Install the Python MongoDB driver
exist Python used in MongoDB Drivers for accessing the database must be installed before：
wget -O /etc// /repo/Centos•
wget -O /etc// /repo/
yum install -y python3
pip3 install pymongo
Import pymongo in python interactive mode and check whether the driver is installed correctly:
import pymongo
[root@lss ~]# python3
Python 3.6.8 (default, Nov 16 2020, 16:55:22)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymongo
>>>
'4.0.2'
>>>
2.Create a connection
Sure MongoDB The connection string uses the driver to connect to MongoDB The cluster only needs to specify MongoDB Just connect the string。Its basic format can be
to reference documentation: Connection String URI Format The simplest form ismongodb://Database server host address：port number
like：mongodb://127.0.0.1:27017
● Initialize database connection
from pymongo import MongoClient
uri = "mongodb://root:[email protected]:27017"
client = MongoClient(uri)
client
3. Database operation: insert data
Initialize database and collections
db = client["eshop"]
user_coll = db["users"]
Insert a new user data
new_user = {"username": "nina", "password": "xxxx", "email":"123456@ "}
result = user_coll.insert_one(new_user)
result
[root@lss ~]# mongo -u root -p root123 admin
> use eshop
switched to db eshop
> show tables;
users
> ()
{ "_id" : ObjectId("6235e1bd6f009bc34f5a7797"), "username" : "nina", "password" : "xxxx", "email" : "123456@ " }
RS(Replica Set)
1.The role of replica set
MongoDB The main significance of replica sets is to achieve high availability of services,Its reality depends on the functionality of two aspects：
• Quickly copy data to another independent node as it is written
• Automatically elect a new replacement node when the node receiving writes fails
While achieving high availability，Replica sets implement several other additional effects：
• data distribution：Copy data from one region to another，Reduce read latency in another region
• Read and write separation：Different types of pressure are executed on different nodes.
• Remote disaster recovery：Quickly switch to an offsite location when the data center fails
2. Typical replica set structure
PSS Architecture：One master and two slaves（3machine）
PSA Architecture：One master, one slave, one vote（3machine）
A typical replica set consists of3Composed of more than one node with voting rights，include：
• a master node（PRIMARY）：Accept write operations and vote during elections
• two（or more）slave node（SECONDARY）：Replicate new data on the master node and vote during elections
• Arbiter（voting node）
3. How data is copied
When a modification operation，Whether inserted、update or delete，When reaching the master node，Its operations on data will be recorded（After some necessary conversions），These records are called oplog。
The slave node opens a tailable The cursor continuously acquires new entries into the main node. oplog，and play it back on your own data，This keeps the data consistent with the master node。
4. Complete fault recovery through elections
Nodes with voting rights send heartbeats to each other.；
● when5When the heartbeat is not received, the node is judged to be disconnected.；
● If the lost connection is the master node，The slave node will initiate an election，Select a new master node；
● If the slave node loses contact, no new election will be generated.；
● The election is based on RAFT Consistency algorithm implementation，The necessary condition for a successful election is that the majority of voting nodes survive；
5. Factors affecting elections
The entire cluster must have a majority of nodes alive；The node that is elected as the master node must：
• Able to establish connections with most nodes
• with newer oplog
• have higher priority（If configured）
6. Common options
Replica set nodes have the following common options:：
• Have voting rights?（v parameter）：Some participate in voting；
• priority（priority parameter）：priority越高的节点越优先成为主节点。priority为0The node cannot become the master node；
• hide（hidden parameter）：Copy data，but not visible to the app。hide节点可以具有投票仅，But the priority must be0；
• Delay（slaveDelay parameter）：copy n Data seconds ago，Keep the time difference with the master node。
7. Notes on replica sets
● About hardware：
• Because any normal replica set node may become the primary node，Their status is the same，Therefore, the hardware configuration must be consistent；
• In order to ensure that the nodes will not go down at the same time，The hardware used by each node must be independent。
● About the software：
• The software version of each node in the replication set must be consistent，to avoid unpredictable problems。
● Adding nodes will not increase system write performance！
Detailed explanation of Set configuration process
1.Planning
three or moremongodbnode（or multiple instances）
multiple instances：
（1）multiple ports：28017、28018、28019、28020
（2）multiple catalogs：
su - mongod
mkdir -p /mongodb/28017/conf /mongodb/28017/data /mongodb/28017/log
mkdir -p /mongodb/28018/conf /mongodb/28018/data /mongodb/28018/log
mkdir -p /mongodb/28019/conf /mongodb/28019/data /mongodb/28019/log
mkdir -p /mongodb/28020/conf /mongodb/28020/data /mongodb/28020/log
(3) Configuration file content preparation
cat > /mongodb/28017/conf/ <<EOF
systemLog:
destination: file
path: /mongodb/28017/log/
logAppend: true
storage:
journal:
enabled: true
dbPath: /mongodb/28017/data
directoryPerDB: true
#engine: wiredTiger engine information
wiredTiger:
engineConfig:
cacheSizeGB: 1
directoryForIndexes: true
collectionConfig:
blockCompressor: zlib
indexConfig:
prefixCompression: true
processManagement:
fork: true
net:
port: 28017
bindIp: 10.0.0.150,127.0.0.1
replication:
oplogSizeMB: 2048
replSetName: my_repl #Cluster name (increases and decreases are all connected to the name)
EOF
\cp /mongodb/28017/conf/ /mongodb/28018/conf/
\cp /mongodb/28017/conf/ /mongodb/28019/conf/
\cp /mongodb/28017/conf/ /mongodb/28020/conf/
sed 's#28017#28018#g' /mongodb/28018/conf/ -i
sed 's#28017#28019#g' /mongodb/28019/conf/ -i
sed 's#28017#28020#g' /mongodb/28020/conf/ -i
------------------------------------------------------------
(4) Start multiple instances for backup
mongod -f /mongodb/28017/conf/
mongod -f /mongodb/28018/conf/
mongod -f /mongodb/28019/conf/
mongod -f /mongodb/28020/conf/
2. Configure replication set (two methods PSS/PSA)
（1）Connect to previous node
mongo --port 28017/mongo --port 28018
use admin
(2) 1 master and 2 slaves, slave library ordinary slave library (PSS)
config = {_id: 'my_repl', members: [
{_id: 0, host: '10.0.0.150:28017'},
{_id: 1, host: '10.0.0.150:28018'},
{_id: 2, host: '10.0.0.150:28019'}]
}
(config)
my_repl:PRIMARY> #Master node
my_repl:SECONDARY> # is the slave node
(3) 1 master, 1 slave, 1 arbiter (PSA)
config = {_id: 'my_repl', members: [
{_id: 0, host: '10.0.0.150:28017'},
{_id: 1, host: '10.0.0.150:28018'},
{_id: 2, host: '10.0.0.150:28019',"arbiterOnly":true}]
}
(config)
3. Replica set testing
#The main database creates a table forMovies，By default，Only the main library can read and write，No reading or writing can be done from the library，Unless set
#Create a test library first
use test
([ { "title" : "Jaws", "year" : 1975,
"imdb_rating" : 8.1 },
... { "title" : "Batman", "year" : 1989, "imdb_rating" : 7.6 },
... ] );
BulkWriteResult({
"writeErrors" : [ ],
"writeConcernErrors" : [ ],
"nInserted" : 2,
"nUpserted" : 0,
"nMatched" : 0,
"nModified" : 0,
"nRemoved" : 0,
"upserted" : [ ]
})
my_repl:PRIMARY> show tables;
movies
#Query whether the slave database has relevant information to verify whether the master-slave replication is successful.
Note: In the mongodb replica set, the slave library does not allow reading and writing by default.
(); #Open the function of reading from the library
my_repl:SECONDARY> ().pretty();
my_repl:SECONDARY> ();
my_repl:SECONDARY> show tables;
movies
my_repl:SECONDARY> ()
{ "_id" : ObjectId("6236ccb0010577ebf1729fad"), "title" : "Jaws", "year" : 1975, "imdb_rating" : 8.1 }
{ "_id" : ObjectId("6236ccb0010577ebf1729fae"), "title" : "Batman", "year" : 1989, "imdb_rating" : 7.6 }
4. Replication set management operations
#（1）View replica set status：
(); //View overall replica set status
############################################
my_repl:PRIMARY> ()
{
"set" : "my_repl",
"date" : ISODate("2022-03-20T07:38:45.198Z"),
"myState" : 1,
"term" : NumberLong(1),
"syncingTo" : "",
"syncSourceHost" : "",
"syncSourceId" : -1,
"heartbeatIntervalMillis" : NumberLong(2000),
"majorityVoteCount" : 2,
"writeMajorityCount" : 2,
"optimes" : {
"lastCommittedOpTime" : {
"ts" : Timestamp(1647761922, 1),
"t" : NumberLong(1)
},
"lastCommittedWallTime" : ISODate("2022-03-20T07:38:42.135Z"),
"readConcernMajorityOpTime" : {
"ts" : Timestamp(1647761922, 1),
"t" : NumberLong(1)
},
"readConcernMajorityWallTime" : ISODate("2022-03-20T07:38:42.135Z"),
"appliedOpTime" : {
"ts" : Timestamp(1647761922, 1),
"t" : NumberLong(1)
},
"durableOpTime" : {
"ts" : Timestamp(1647761922, 1),
"t" : NumberLong(1)
},
"lastAppliedWallTime" : ISODate("2022-03-20T07:38:42.135Z"),
"lastDurableWallTime" : ISODate("2022-03-20T07:38:42.135Z")
},
"lastStableRecoveryTimestamp" : Timestamp(1647761912, 1),
"lastStableCheckpointTimestamp" : Timestamp(1647761912, 1),
"electionCandidateMetrics" : {
"lastElectionReason" : "electionTimeout",
"lastElectionDate" : ISODate("2022-03-20T05:46:30.828Z"),
"electionTerm" : NumberLong(1),
"lastCommittedOpTimeAtElection" : {
"ts" : Timestamp(0, 0),
"t" : NumberLong(-1)
},
"lastSeenOpTimeAtElection" : {
"ts" : Timestamp(1647755179, 1),
"t" : NumberLong(-1)
},
"numVotesNeeded" : 2,
"priorityAtElection" : 1,
"electionTimeoutMillis" : NumberLong(10000),
"numCatchUpOps" : NumberLong(0),
"newTermStartDate" : ISODate("2022-03-20T05:46:30.844Z"),
"wMajorityWriteAvailabilityDate" : ISODate("2022-03-20T05:46:31.989Z")
},
"members" : [
{
"_id" : 0, #node1
"name" : "10.0.0.150:28017",
"health" : 1, #health condition
"state" : 1, #state=1 is the master node, state=2 is the slave node
"stateStr" : "PRIMARY",
"uptime" : 13371,
"optime" : { #Log version information, you can check whether the slave database information has arrived. Once it arrives, it means the synchronization is completed.
"ts" : Timestamp(1647761922, 1),
"t" : NumberLong(1)
},
"optimeDate" : ISODate("2022-03-20T07:38:42Z"),
"syncingTo" : "",
"syncSourceHost" : "",