elasticsearch 集群搭建

####环境
三节点es集群： 192.168.10.91
192.168.10.92
192.168.10.93

####安装elasticsearch

# tar xzvf elasticsearch-1.6.0.tar.gz

# cd elasticsearch-1.6.0

修改配置

# cat config/elasticsearch.yml |grep -v ^$|grep -v ^#

cluster.name: test-es-cluster     #每个节点一致
node.name: node1                  #每个节点的name，每个节点不一致
bootstrap.mlockall: true          #直接占用所有内存
transport.tcp.port: 9300
http.port: 9200
discovery.zen.minimum_master_nodes: 2          #集群中最小节点数，5节点为3，3节点为2
discovery.zen.ping.timeout: 30s
discovery.zen.ping.unicast.hosts: ["192.168.10.91", "192.168.10.92","192.168.10.93"]   #每个node的ip：port

安装head插件

# git clone https://github.com/mobz/elasticsearch-head.git

# mkdir mobz

# mv elasticsearch-head mobz/

# ./bin/plugin -install mobz/elasticsearch-head/

-> Installing mobz/elasticsearch-head/...
Trying https://github.com/mobz/elasticsearch-head/archive/master.zip... Downloading .............................................................................................................................................................................................................................................................DONE
Installed mobz/elasticsearch-head/ into /data/packages/elasticsearch-1.6.0/plugins/head

-安装bigdesk插件

# mkdir lukas-vlcek

# mv bigdesk lukas-vlcek/

# ./bin/plugin -install lukas-vlcek/bigdesk/

-> Installing lukas-vlcek/bigdesk/...
Trying https://github.com/lukas-vlcek/bigdesk/archive/master.zip... Downloading .........................................................................................................................................................................................................................................................DONE
Installed lukas-vlcek/bigdesk/ into /data/packages/elasticsearch-1.6.0/plugins/bigdesk
Identified as a _site plugin, moving to _site structure ...

安装ik分词插件

# git clone https://github.com/medcl/elasticsearch-analysis-ik.git

# cd elasticsearch-analysis-ik/

# mvn package

...

# ./../bin/plugin -install analysis-ik --url file:///data/packages/elasticsearch-1.6.0/elasticsearch-analysis-ik/target/releases/elasticsearch-analysis-ik-1.9.1.zip

-> Installing analysis-ik...
Trying file:/data/packages/elasticsearch-1.6.0/elasticsearch-analysis-ik/target/releases/elasticsearch-analysis-ik-1.9.1.zip...
Downloading ............................................DONE
Installed analysis-ik into /data/packages/elasticsearch-1.6.0/plugins/analysis-ik

# cp -a config/ik ../config/

# vi ../config/elasticsearch.yml

在尾部添加如下内容

index:
analysis:
analyzer:
ik:
alias: [ik_analyzer]
type: org.elasticsearch.index.analysis.IkAnalyzerProvider
ik_max_word:
type: ik
use_smart: false
ik_smart:
type: ik
use_smart: true

启动elasticsearch

# cp -a /data/packages/elasticsearch-1.6.0 /data/app/

# /data/app/elasticsearch-1.6.0/bin/elasticsearch -d

-查看head插件

http://192.168.10.91:9200/_plugin/head/



查看bigdesk插件

http://192.168.10.91:9200/_plugin/bigdesk/#nodes

####elasticsearch优化

1：file descriptors
调整系统允许用户打开文件数的最大值，通过命令：
curl -XGET http://localhost:9200/_nodes/process?pretty
“max_file_descriptors” : 4096,字段表明当前系统的值。
此值在/etc/security/limits.conf 中可以进行调整
2：virtual memory
Elasticsearch 默认使用 hybrid mmap / nio fs的方式来存储。
用这种方式如果系统vm.max_map_count值偏小会导致内存溢出。且此参数影响创建线程的数量
vim /etc/sysctl.conf
vm.max_map_count=262144

sysctl -p
【 hybrid mmap / nio fs ：混合mmap和nio fs的存储方式】
3：memory settings
关闭swap：如果可以，让Elasticsearch单独跑在一台机器，其内存使用量完全由ES_HEAP_SIZE变量控制。如果Elasticsearch使用了swap，会导致性能下降。
1）swapoff -a
2）vim /etc/sysctl.conf
vm.swappiness=0
3）vim elasticsearch.yml
bootstrap.mlockall: true
如果curl -XGET http://localhost:9200/_nodes/process?pretty 显示bootstrap.mlockall还是为false
1：运行elasticsearch的用户没有权限锁内存
2：/tmp/ 使用noexec的挂载选项，解决办法：./bin/elasticsearch -Djna.tmpdir=/path/to/new/dir
4：避免脑裂
discovery.zen.minimum_master_nodes=(num_master_nodes/2)+1
ping_timeout默认30s
ping_retries默认3次
5、如果使用的内存超过6G使用g1垃圾收集
注释下面四行
JAVA_OPTS=”$JAVA_OPTS -XX:+UseParNewGC”
JAVA_OPTS=”$JAVA_OPTS -XX:+UseConcMarkSweepGC”
JAVA_OPTS=”$JAVA_OPTS -XX:CMSInitiatingOccupancyFraction=75″
JAVA_OPTS=”$JAVA_OPTS -XX:+UseCMSInitiatingOccupancyOnly”
然后换成：
JAVA_OPTS=”$JAVA_OPTS -XX:+UseG1GC”
JAVA_OPTS=”$JAVA_OPTS -XX:MaxGCPauseMillis=200″

es操作

添加index
# curl -XPOST http://192.168.10.91:9200/index1

{"acknowledged":true}

# curl -XPOST http://192.168.10.91:9200/index2

{"acknowledged":true}

# curl -XPOST http://192.168.10.91:9200/index3

{"acknowledged":true}



删除index

# curl -XDELETE http://192.168.10.91:9200/index3

{"acknowledged":true}

# curl -XDELETE http://192.168.10.91:9200/index* #删除多index

{"acknowledged":true}

添加一条记录

# curl -XPOST -d '{"name":"gongjun","age":"2"}' http://192.168.10.91:9200/index1/type/1

{"_index":"index1","_type":"type","_id":"1","_version":1,"created":true}



删除一条记录

# curl -XDELETE http://192.168.10.91:9200/index1/type/1

{"found":true,"_index":"index1","_type":"type","_id":"1","_version":2}

获取一条数据

# curl http://192.168.10.91:9200/index1/type/2

{"_index":"index1","_type":"type","_id":"2","_version":1,"found":true,"_source":{"name":"gongjun","age":"25"}}

查询index1下数据

# curl http://192.168.10.91:9200/index1/_search?pretty=true

{
"took" : 17,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 2,
"max_score" : 1.0,
"hits" : [ {
"_index" : "index1",
"_type" : "type2",
"_id" : "1",
"_score" : 1.0,
"_source":{"name":"gongjun","age":"25"}
}, {
"_index" : "index1",
"_type" : "type",
"_id" : "2",
"_score" : 1.0,
"_source":{"name":"gongjun","age":"25"}
} ]
}
}

查询index1下type下的所有数据

# curl http://192.168.10.91:9200/index1/type/_search?pretty=true

{
"took" : 9,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [ {
"_index" : "index1",
"_type" : "type",
"_id" : "2",
"_score" : 1.0,
"_source":{"name":"gongjun","age":"25"}
} ]
}
}

查看node信息

# curl http://192.168.10.91:9200/

{
"status" : 200,
"name" : "node1",
"cluster_name" : "test-es-cluster",
"version" : {
"number" : "1.6.0",
"build_hash" : "cdd3ac4dde4f69524ec0a14de3828cb95bbb86d0",
"build_timestamp" : "2015-06-09T13:36:34Z",
"build_snapshot" : false,
"lucene_version" : "4.10.4"
},
"tagline" : "You Know, for Search"
}

查看集群状态

# curl http://192.168.10.91:9200/_cluster/health?pretty=true

{
"cluster_name" : "test-es-cluster",
"status" : "green",
"timed_out" : false,
"number_of_nodes" : 3,
"number_of_data_nodes" : 3,
"active_primary_shards" : 5,
"active_shards" : 10,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch" : 0
}