storm集群搭建
2016-06-03 21:14
465 查看
安装环境:
虚拟操作系统:centOS7 64位 2台
IP地址为192.168.1.10 192.168.1.11
内存:1GB内存
zookeeper版本 :3.4.6
Storm版本:0.9.6
安装步骤:
1、安装JDK等步骤省略。
1、下载storm安装包,解压。
1、配置/apache-storm-0.9.6/conf/storm.yaml配置文件
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
########### These MUST be filled in for a storm configuration
storm.zookeeper.servers:
- "192.168.1.10"
- "192.168.1.11"
nimbus.host : "192.168.1.10"
storm.local.dir : "/opt/stormtmp/data"
ui.port : 8080
supervisor.slots.ports :
-6700
-6701
-6702
-6703
#
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
# - org.mycompany.MyType
# - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
# - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
# - "server1"
# - "server2"
## Metrics Consumers
# topology.metrics.consumer.register:
# - class: "backtype.storm.metric.LoggingMetricsConsumer"
# parallelism.hint: 1
# - class: "org.mycompany.MyMetricsConsumer"
# parallelism.hint: 1
# argument:
# - endpoint: "metrics-collector.mycompany.org"
配置详解:
storm.zookeeper.servers: ZooKeeper服务器列表
supervisor.slots.ports :且每个端口只运行一个worker.通过这项配置可以调整每台机器上运行的worker数.(调整slot数/每机)
Storm.local.dir:storm使用的本地文件系统目录(必须存在并且storm进程可读写)
Nimbus.host:nimbus服务器地址
Ui.port:Storm UI的服务端口
nimbus.host :nimbus服务器地址
2、启动storm。先启动10的storm。
./storm nimbus &
./storm supervisor &
./storm ui &
IP为11结尾的机器运行
./storm supervisor &
3、查看运行状态,访问localhost:8080,进入storm ui界面,查看集群成功。
虚拟操作系统:centOS7 64位 2台
IP地址为192.168.1.10 192.168.1.11
内存:1GB内存
zookeeper版本 :3.4.6
Storm版本:0.9.6
安装步骤:
1、安装JDK等步骤省略。
1、下载storm安装包,解压。
1、配置/apache-storm-0.9.6/conf/storm.yaml配置文件
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
########### These MUST be filled in for a storm configuration
storm.zookeeper.servers:
- "192.168.1.10"
- "192.168.1.11"
nimbus.host : "192.168.1.10"
storm.local.dir : "/opt/stormtmp/data"
ui.port : 8080
supervisor.slots.ports :
-6700
-6701
-6702
-6703
#
#
# ##### These may optionally be filled in:
#
## List of custom serializations
# topology.kryo.register:
# - org.mycompany.MyType
# - org.mycompany.MyType2: org.mycompany.MyType2Serializer
#
## List of custom kryo decorators
# topology.kryo.decorators:
# - org.mycompany.MyDecorator
#
## Locations of the drpc servers
# drpc.servers:
# - "server1"
# - "server2"
## Metrics Consumers
# topology.metrics.consumer.register:
# - class: "backtype.storm.metric.LoggingMetricsConsumer"
# parallelism.hint: 1
# - class: "org.mycompany.MyMetricsConsumer"
# parallelism.hint: 1
# argument:
# - endpoint: "metrics-collector.mycompany.org"
配置详解:
storm.zookeeper.servers: ZooKeeper服务器列表
supervisor.slots.ports :且每个端口只运行一个worker.通过这项配置可以调整每台机器上运行的worker数.(调整slot数/每机)
Storm.local.dir:storm使用的本地文件系统目录(必须存在并且storm进程可读写)
Nimbus.host:nimbus服务器地址
Ui.port:Storm UI的服务端口
nimbus.host :nimbus服务器地址
2、启动storm。先启动10的storm。
./storm nimbus &
./storm supervisor &
./storm ui &
IP为11结尾的机器运行
./storm supervisor &
3、查看运行状态,访问localhost:8080,进入storm ui界面,查看集群成功。
相关文章推荐
- https相关原理
- 查找两个数组的公共元素
- 观察者模式
- linux socket 编程
- 读软件工程的职业道德有感
- 由linux命令行下的两条命令来阐释输出结果和返回值的区别
- iOS的动画效果类型及实现方法
- 华为项目管理10大模板Excel版(可直接套用_非常实用)
- 0603 学术诚信与道德
- 学术诚信与职业道德的思考
- zookeeper集群搭建
- 有感而发
- MXNet 的学习(一)—— MXNet Dependency Engine(依赖引擎)
- VC6和VS2005(VC8)各项目默认运行时库
- JVM调优系列(二)——JVM内存分配解析
- 简单倒计时效果
- php 简单操作数据库
- EasyTouch的使用
- 最大子矩阵
- C primer plus 第十章 练习9: