storm on yarn(CDH5) 部署笔记
按照storm on yarn(Apache hadoop)部署好之后,然后修改HADOOP_HOME,hadoopenv.sh中的JAVA_HOME,以及storm-yarn-master中pom.xml的hadoop version
启动顺序依次为:
hdfs:start-dfs.sh
yarn:start-yarn.sh
zookeeper:zkServer.sh start
storm
nimbus:storm nimbus
UI:storm ui #localhost:8080
supervisor:storm supervisor 如果有从节点的话
提交storm任务:
storm-yarn launch storm.yaml
存储storm.yaml文件到.storm,这样使用storm命令提交jobs时,会找到该文件 ---我的这一步出错,但是在apache hadoop下成功了
storm-yarn getStormConfig -appId application_1381089732797_0025 -output ~/.storm/storm.yaml
提交Topology
storm jar lib/storm-starter-0.0.1-SNAPSHOT.jar storm.starter.WordCountTopology WordCountTopology -c nimbus.host=<your nimbus host>
监控Topology
查看Storm ui,地址是:http://<your nimbus host>:8080
关闭Topology
storm kill [Topology_name]
关闭Storm on yarn集群
storm-yarn shutdown –appId [applicationId]
致谢:http://blog.csdn.net/m_vptr/article/details/9962613
错误:15/07/19 00:31:46 INFO yarn.StormOnYarn: application report for application_1437236195968_0001 :N/A:-1
15/07/19 00:31:46 INFO yarn.StormOnYarn: Attaching to N/A:-1 to talk to app master application_1437236195968_0001
Exception in thread "main" java.lang.IllegalArgumentException: invalid port: -1
待解决
转载于:https://www.cnblogs.com/prisoner/p/4657149.html
- storm on yarn(CDH5) 部署笔记
- Storm On Yarn 安装部署
- storm on yarn 最简单安装笔记!!!私货哦!!!
- storm on yarn 部署
- storm on yarn 部署
- Storm之——Storm on Yarn 部署
- Storm on YARN 安装部署与感悟
- Storm on yarn安装
- Spark1.0.0 on YARN 模式部署
- storm on Yarn
- Spark2.0.1 on yarn with hue 集群安装部署(六)hue+hive+sparksql
- Spark On YARN 集群安装部署
- spark集群安装部署(spark on yarn)
- flink on yarn部署
- 【转载】Spark On YARN 集群安装部署
- storm-on-yarn配置
- Spark On YARN 集群安装部署
- storm on yarn 如何支持 storm 0.9.2
- Storm On YARN
- Storm on YARN