Mongodb简单的分片搭建与测试

Mongodb简单的分片搭建与测试

环境准备：

服务器：

192.168.0.126 CentOS release 6.5 (Final)

192.168.0.136 CentOS release 6.5 (Final)

Mongodb版本：

mongodb-linux-x86_64-rhel62-3.0.5.tgz

分片介绍：

分片(sharding)其实就是数据拆分，把数据分散在多个节点上，也就是水平拆分。MongoDB支持自动分片，无论自动分片有多优点或缺点，MongoDB依然拥有该特性而引以为傲。

MongoDB分片适用于如下几个场景：

1.单个服务器无法承受压力时，压力包括负载、频繁写、吞吐量等；

2.服务器磁盘空间不足时；

3.增加可用内存大小，以更多的数据在内存中访问。

要构建一个 MongoDB Sharding Cluster，需要三种角色：

Shard Server

即存储实际数据的分片，每个Shard可以是一个mongod实例，也可以是一组mongod实例构成的Replica Set。为了实现每个Shard内部的auto-failover，MongoDB官方建议每个Shard为一组Replica Set。

Config Server

为了将一个特定的collection存储在多个shard中，需要为该collection指定一个shard key，例如{age: 1} ，shard key可以决定该条记录属于哪个chunk。Config Servers就是用来存储：所有shard节点的配置信息、每个chunk的shard key范围、chunk在各shard的分布情况、该集群中所有DB和collection的sharding配置信息。

Route Process

这是一个前端路由，客户端由此接入，然后询问Config Servers需要到哪个Shard上查询或保存记录，再连接相应的Shard进行操作，最后将结果返回给客户端。客户端只需要将原本发给mongod的查询或更新请求原封不动地发给Routing Process，而不必关心所操作的记录存储在哪个Shard上。

Mongodb分片环境搭建：

1.规划：

两台机器上构建一个集群分片：

规划对应的端口号：

192.168.0.126 Shard Server 1：27017

192.168.0.136 Shard Server 2：27018

192.168.0.136 Config Server ：30000

192.168.0.136 Route Process：40000

2.在服务器上创建目录：

192.168.0.126：

mkdir -p /usr/local/mongodb/data/shard/s0 --创建数据目录

mkdir -p /usr/local/mongodb/data/shard/log --创建日志目录

192.168.0.136：

mkdir -p /opt/mongodb/data/shard/s1 --创建数据目录

mkdir -p /opt/mongodb/data/shard/log --创建日志目录

上传mongdb安装文件到/usr/local/mongodb和/opt/mongodb/目录解压

tar -zxvf mongodb-linux-x86_64-rhel62-3.0.5.tgz

然后把解压的bin目录下的所有文件移动/usr/local/mongodb和/opt/mongodb/目录下

3.启动Shard Server实例

启动Shard Server实例1（192.168.0.126）

/usr/local/mongodb/mongod --port 27017 --dbpath /usr/local/mongodb/data/shard/s0 --fork --logpath /usr/local/mongodb/data/shard/log/s0.log

启动Shard Server实例2（192.168.0.136）

/opt/mongodb/mongod --port 27018 --dbpath /opt/mongodb/data/shard/s1 --fork --logpath /opt/mongodb/data/shard/log/s1.log

4. 启动Config Server（192.168.0.136）

创建数据目录:

mkdir -p /opt/mongodb/data/shard/config

/opt/mongodb/mongod --port 30000 –dbpath /opt/mongodb/data/shard/config --fork --logpath /opt/mongodb/data/shard/log/config.log

4. 启动Route Process（192.168.0.136）

/opt/mongodb/mongos --port 40000 --configdb 192.168.0.136:30000 --fork --logpath /opt/mongodb/data/shard/log/route.log --chunkSize 1 --启动Route Server实例

5. 配置Sharding

[root@tts log]# /opt/mongodb/mongo admin --port 40000

MongoDB shell version: 3.0.5

connecting to: 127.0.0.1:40000/admin

Welcome to the MongoDB shell.

For interactive help, type "help".

For more comprehensive documentation, see
http://docs.mongodb.org/
Questions? Try the support group
http://groups.google.com/group/mongodb-user
Server has startup warnings:

2015-08-06T10:38:46.608+0800 I CONTROL ** WARNING: You are running this process as the root user, which is not recommended.

2015-08-06T10:38:46.608+0800 I CONTROL

mongos>

mongos>

mongos>

mongos>

mongos> db.runCommand({ addshard:"192.168.0.126:27017" })db.runCommand({ addshard:"192.168.0.126:27017" }) -----添加 Shard Server（分片1）

{ "shardAdded" : "shard0000", "ok" : 1 }

mongos> db.runCommand({ addshard:"192.168.0.136:27018" })db.runCommand({ addshard:"192.168.0.136:27018" }) -----添加 Shard Server（分片2）

{ "shardAdded" : "shard0001", "ok" : 1 }

mongos> db.runCommand({ enablesharding:"test" })db.runCommand({ enablesharding:"test" }) ----设置分片存储的数据库

{ "ok" : 1 }

测试分片：

> db.runCommand({ shardcollection: "test.users", key: { id:1 }}) --设置分片的集合名称。且必须指定Shard Key，系统会自动创建索引

{ "collectionsharded" : "test.users", "ok" : 1 }

--循环插入数据测试

mongos > use test

switched to db test

mongos> for (var i = 1; i <= 500000; i++) db.users.insert({age:i, name:"wangwenlong", addr:"Beijing", country:"China"})for (var i = 1; i <= 500000; i++) db.users.insert({age:i, name:"wangwenlong", addr:"Beijing", country:"China"})

WriteResult({ "nInserted" : 1 })

mongos> db.users.stats() db.users.stats()

{

"sharded" : true, --说明此表已被 shard

"paddingFactorNote" : "paddingFactor is unused and unmaintained in 3.0. It remains hard coded to 1.0 for compatibility only.",

"userFlags" : 1,

"capped" : false,

"ns" : "test.users",

"count" : 500000,

"numExtents" : 16,

"size" : 56000000,

"storageSize" : 75595776,

"totalIndexSize" : 16294768,

"indexSizes" : {

"_id_" : 16294768

},

"avgObjSize" : 112,

"nindexes" : 1,

"nchunks" : 99,

"shards" : {

"shard0000" : { --在此分片实例上约有251038条记录

"ns" : "test.users",

"ns" : "test.users",

"count" : 251038,

"size" : 28116256,

"avgObjSize" : 112,

"numExtents" : 8,

"storageSize" : 37797888,

"lastExtentSize" : 15290368,

"paddingFactor" : 1,

"paddingFactorNote" : "paddingFactor is unused and unmaintained in 3.0. It remains hard coded to 1.0 for compatibility only.",

"userFlags" : 1,

"capped" : false,

"nindexes" : 1,

"totalIndexSize" : 8176000,

"indexSizes" : {

"_id_" : 8176000

},

"ok" : 1

},

"shard0001" : { --在此分片实例上约有248962条记录

"ns" : "test.users",

"count" : 248962,

"size" : 27883744,

"avgObjSize" : 112,

"numExtents" : 8,

"storageSize" : 37797888,

"lastExtentSize" : 15290368,

"paddingFactor" : 1,

"paddingFactorNote" : "paddingFactor is unused and unmaintained in 3.0. It remains hard coded to 1.0 for compatibility only.",

"userFlags" : 1,

"capped" : false,

"nindexes" : 1,

"totalIndexSize" : 8118768,

"indexSizes" : {

"_id_" : 8118768

},

"ok" : 1

}

},

"ok" : 1

}

列出所有的 Shard Server（分片）

mongos> db.runCommand({ listshards: 1 }) db.runCommand({ listshards: 1 })

{

"shards" : [

{

"_id" : "shard0000",

"host" : "192.168.0.126:27017"

},

{

"_id" : "shard0001",

"host" : "192.168.0.136:27018"

}

],

"ok" : 1

}

查看 Sharding 信息

mongos> printShardingStatus()

--- Sharding Status ---

sharding version: {

"_id" : 1,

"minCompatibleVersion" : 5,

"currentVersion" : 6,

"clusterId" : ObjectId("55c2c8b693ea321a1f4dd128")

}

shards:

{ "_id" : "shard0000", "host" : "192.168.0.126:27017" }

{ "_id" : "shard0001", "host" : "192.168.0.136:27018" }

balancer:

Currently enabled: yes

Currently running: no

Failed balancer rounds in last 5 attempts: 0

Migration Results for the last 24 hours:

49 : Success

databases:

{ "_id" : "admin", "partitioned" : false, "primary" : "config" }

{ "_id" : "test", "partitioned" : true, "primary" : "shard0000" }

test.users

shard key: { "_id" : 1 }

chunks:

shard0000 50

shard0001 49

too many chunks to print, use verbose if you want to force print

mongos>