elasticsearch搭建及python简单使用

这系列文章主要记录自己学习Elasticsearch的过程，因为项目组es的使用量日益增加，作为一名平凡的运维非专业开发人员，是有必要仔细的从头再过一次这些基本的概念。也是作为Blog的第一篇文章，像众多大牛学习。

环境描述：

oracle virtual Box 虚拟机

centos7.3

双网卡，10.0.0.2 主机访问，通讯接口

elasticsearch安装

在es的官网上下载最近的安装包：

https://www.elastic.co/downloads/elasticsearch

目前最近版本Version:5.6.3，公司使用的版本的是5.4.2。

上传到服务器上后解压：

tar zxvf elasticsearch-5.6.3.tar.gz

修改config目录下文件，修改data文件和logs文件目录及绑定的IP地址：

[root@bogon config]# pwd

/data01/elasticsearch-5.6.3/config

[root@bogon config]# ls

elasticsearch.yml jvm.options log4j2.properties scripts

插入以下配置

path.data: /data01/elasticsearch-5.6.3/data

path.logs: /data01/elasticsearch-5.6.3/logs

network.host: 10.0.0.2

es配置请参考以下url：

https://www.elastic.co/guide/en/elasticsearch/reference/master/settings.html

修改完成后，切换到其他用户启动，如果是用 root目录启动，再日志文件中会有报错

java.lang.RuntimeException: can not run elasticsearch as root

at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:106) ~[elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:195) ~[elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:342) [elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:132) [elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:123) [elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:70) [elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) [elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) [elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) [elasticsearch-5.6.3.jar:5.6.3]

[2017-10-21T10:01:31,085][WARN ][o.e.b.ElasticsearchUncaughtExceptionHandler] [docker-1] uncaught exception in thread [main]

org.elasticsearch.bootstrap.StartupException: java.lang.RuntimeException: can not run elasticsearch as root

at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136) ~[elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:123) ~[elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:70) ~[elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134) ~[elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.cli.Command.main(Command.java:90) ~[elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91) ~[elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84) ~[elasticsearch-5.6.3.jar:5.6.3]

Caused by: java.lang.RuntimeException: can not run elasticsearch as root

at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:106) ~[elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:195) ~[elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:342) ~[elasticsearch-5.6.3.jar:5.6.3]

at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:132) ~[elasticsearch-5.6.3.jar:5.6.3]

… 6 more

报错的原因是el默认不能使用root启动。

解决方案：

将es目录授权给其他用户：

chown -R test:test /data01/elasticsearch-5.6.3

su - test

/data01/elasticsearch-5.6.3/bin

以下是elasticsearch命令的帮助部分，通过执行elasticsearch -h可以查看。

starts elasticsearch

Option Description

—— ———–

-E Configure a setting

-V, –version Prints elasticsearch version information and exits 查看版本

-d, –daemonize Starts Elasticsearch in the background 后台运
8fd6
维

-h, –help show help 查看帮助

-p, –pidfile Creates a pid file in the specified path on start 文件PID位置

-q, –quiet Turns off standard ouput/error streams logging in console 安静模式

-s, –silent show minimal output 最小化输出

-v, –verbose show verbose output 可视化模式

启动es

./elasticsearch -d -p /data01/elasticsearch-5.6.3/logs/es.pid

启动报错解决方案：

[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536]

[2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

以上两个跑错主要是因为系统参数没有优化导致的。

第一个报错，在 /etc/security/limits.conf中添加以下两行：

* soft nofile 65536

* hard nofile 65536

第二个报错，在/etc/sysctl.conf添加：

vm.max_map_count = 262144

再次启动， 查看日志，正常启动。

测试

浏览器中输入

http://10.0.0.2:9200/_cluster/health?pretty

输出如下，说明ES启动成功。

{

“cluster_name” : “xiafanli”,

“status” : “green”,

“timed_out” : false,

“number_of_nodes” : 1,

“number_of_data_nodes” : 1,

“active_primary_shards” : 0,

“active_shards” : 0,

“relocating_shards” : 0,

“initializing_shards” : 0,

“unassigned_shards” : 0,

“delayed_unassigned_shards” : 0,

“number_of_pending_tasks” : 0,

“number_of_in_flight_fetch” : 0,

“task_max_waiting_in_queue_millis” : 0,

“active_shards_percent_as_number” : 100.0

}

基本查询语句

pythonAPI访问

elasticsearch安装

如果能访问外网的话，可以直接用pip安装

pip install elasticsearch

如果没有连接外网，可以在pypi上查找elasticsearch的包下载解压，用命令行安装：

python setup.py install

基本增删查

# _*_ coding:UTF-8 _*_
from elasticsearch  import Elasticsearch

host = {"host":"10.0.0.2","port":9200}
#初始化
es = Elasticsearch([host])
#创建index
#
es.indices.create(index="test_index_python")