centos的twitter storm安装和storm-start的本地运行
2012-12-25 15:42
501 查看
这里介绍一下storm的基本安装和不用lein的storm-starter运行方法。
一. Storm及相关软件安装
1. 安装python2.7.2
============================
# wget http://www.python.org/ftp/python/2.7.2/Python-2.7.2.tgz # tar zxvf Python-2.7.2.tgz
# cd Python-2.7.2
# ./configure
# make
# make install
# vi /ect/ld.so.conf
--
文件末尾追加
/usr/local/lib
--
#sudo ldconfig
============================
2. 安装zookeeper
============================
# wget http://ftp.meisei-u.ac.jp/mirror/apache/dist//zookeeper/zookeeper-3.3.3/zookeeper-3.3.3.tar.gz # tar zxf zookeeper-3.3.3.tar.gz
# cp -R zookeeper-3.3.3 /usr/local/
# ln -s /usr/local/zookeeper-3.3.3/ /usr/local/zookeeper
设置 ZOOKEEPER_HOME 和$ZOOKEEPER_HOME/bin
#vi ~/.bashrc
--
追加
export ZOOKEEPER_HOME="/path/to/zookeeper"
export PATH=$PATH:$ZOOKEEPER_HOME/bin
--
用zoo_sample.cfg制作$ZOOKEEPER_HOME/conf/zoo.cfg
#cp /usr/local/zookeeper/conf/zoo_sample.cfg /usr/local/zookeeper/conf/zoo.cfg
制作dataLogDir和运行Dir
sudo mkdir /tmp/zookeeper
sudo mkdir /var/log/zookeeper
============================
3. 安装zeromq & jzmq
============================
1)安装zeromq
# wget http://download.zeromq.org/historic/zeromq-2.1.7.tar.gz # tar zxf zeromq-2.1.7.tar.gz
# cd zeromq-2.1.7
# ./configure
**********************
如果报错
configure: error: in `/usr/local/download/zeromq-2.1.7':
configure: error: no acceptable C compiler found in $PATH
See `config.log' for more details
原因为没有安装c compiler
解决方法
# yum install gcc*
如果遇到Error:cannot link with -luuid, install uuid-dev
原因为缺少uuid相关package
解决方法
# yum install uuid*
# yum install e2fsprogs*
# yum install libuuid*
**********************
#make
#make install
更新LD_LIBRARY_PATH
#ldconfig
============================
2)安装jzmq
============================
#yum install git
#git clone git://github.com/nathanmarz/jzmq.git
**********************
如果有proxy的话执行
#git clone http://github.com/nathanmarz/jzmq.git 就能避开git的代理设置
**********************
# cd jzmq
# ./autogen.sh
**********************
如果报错
autogen.sh:
error:
could
not
find
libtool.
libtool is required to run autogen.sh.
缺少libtool
# yum install libtool*
**********************
# ./configure
# make
# make install
============================
4. 安装storm
============================
# wget https://github.com/downloads/nathanmarz/storm/storm-0.7.1.zip # unzip storm- 0.7.1 zip
# cp -R storm-0.7.1 /usr/local/
# vi ~/.bashrc
--
追加
export STORM_HOME=/usr/local/storm-0.7.1
export PATH=$PATH:$STORM_HOME/bin
--
============================
到此为止,storm安装完毕。
二. 本地运行测试程序storm-start
按照https://github.com/nathanmarz/storm-starter,执行这个程序需要用lein,这里介绍的方法用eclipse代替lein的作用。
1) 安装twitter4j
# mkdir twitter4j
# cd twitter4j
# wget http://twitter4j.org/en/twitter4j-2.2.5.zip # unzip twitter4j-2.2.5.zip
2) 追加源文件storm-start/src/jvm/storm
使用eclipse建立java project。追加twitter4j和storm的jar文件。
File-> New -> Java Project ->随便取个名字-> Next -> Libraries -> add External JARs...-> 追加twitter4j和storm的jar文件(/path/to/twitter4j/lib/*.jar和/path/to/storm/lib/*.jar和/path/to/storm/storm-{version}.jar)->
Finsh
导入storm-start
File -> Import -> General -> File System -> Next -> Browse(From directory) -> /path/to/storm-start/src/jvm/storm -> Browse(Info floder)
-> xxx -> src -> OK -> “storm” 和 “Create top-level folder”前打勾 -> Finish
完成之后如图:
3) 追加源文件storm-start/multilang/resources×(python 文件word count用)
File -> Import -> General -> File System -> Next -> Browse(From directory) -> /path/to/storm-start/multilang/resources -> Browse(Info floder)
-> xxx -> OK -> check “resources” and “Create top-level folder” -> Finish
2个源文件都追加好之后,eclipse左边显示如下图:
4) JAR export
File -> Export -> JAR -> JAR file -> 取消 “.classpath” ,“.project” 和 “<.settings” ->的勾 browse -> path/to/export/name.jar -> Finish (忽视 warnings)
4. 执行刚才编译的文件。
# storm jar StormStarter.jar storm.starter.ExclamationTopology
如果出现类似下面的文字,说明运行成功!
....
11367 [Thread-25] INFO
backtype.storm.daemon.task - Emitting: class storm.starter.ExclamationTopology$ExclamationBolt source: 2:3, stream: 1, id: {}, [golda!!!]
....
参考链接:https://github.com/nathanmarz
http://tjun.jp/blog/category/storm/
一. Storm及相关软件安装
1. 安装python2.7.2
============================
# wget http://www.python.org/ftp/python/2.7.2/Python-2.7.2.tgz # tar zxvf Python-2.7.2.tgz
# cd Python-2.7.2
# ./configure
# make
# make install
# vi /ect/ld.so.conf
--
文件末尾追加
/usr/local/lib
--
#sudo ldconfig
============================
2. 安装zookeeper
============================
# wget http://ftp.meisei-u.ac.jp/mirror/apache/dist//zookeeper/zookeeper-3.3.3/zookeeper-3.3.3.tar.gz # tar zxf zookeeper-3.3.3.tar.gz
# cp -R zookeeper-3.3.3 /usr/local/
# ln -s /usr/local/zookeeper-3.3.3/ /usr/local/zookeeper
设置 ZOOKEEPER_HOME 和$ZOOKEEPER_HOME/bin
#vi ~/.bashrc
--
追加
export ZOOKEEPER_HOME="/path/to/zookeeper"
export PATH=$PATH:$ZOOKEEPER_HOME/bin
--
用zoo_sample.cfg制作$ZOOKEEPER_HOME/conf/zoo.cfg
#cp /usr/local/zookeeper/conf/zoo_sample.cfg /usr/local/zookeeper/conf/zoo.cfg
制作dataLogDir和运行Dir
sudo mkdir /tmp/zookeeper
sudo mkdir /var/log/zookeeper
============================
3. 安装zeromq & jzmq
============================
1)安装zeromq
# wget http://download.zeromq.org/historic/zeromq-2.1.7.tar.gz # tar zxf zeromq-2.1.7.tar.gz
# cd zeromq-2.1.7
# ./configure
**********************
如果报错
configure: error: in `/usr/local/download/zeromq-2.1.7':
configure: error: no acceptable C compiler found in $PATH
See `config.log' for more details
原因为没有安装c compiler
解决方法
# yum install gcc*
如果遇到Error:cannot link with -luuid, install uuid-dev
原因为缺少uuid相关package
解决方法
# yum install uuid*
# yum install e2fsprogs*
# yum install libuuid*
**********************
#make
#make install
更新LD_LIBRARY_PATH
#ldconfig
============================
2)安装jzmq
============================
#yum install git
#git clone git://github.com/nathanmarz/jzmq.git
**********************
如果有proxy的话执行
#git clone http://github.com/nathanmarz/jzmq.git 就能避开git的代理设置
**********************
# cd jzmq
# ./autogen.sh
**********************
如果报错
autogen.sh:
error:
could
not
find
libtool.
libtool is required to run autogen.sh.
缺少libtool
# yum install libtool*
**********************
# ./configure
# make
# make install
============================
4. 安装storm
============================
# wget https://github.com/downloads/nathanmarz/storm/storm-0.7.1.zip # unzip storm- 0.7.1 zip
# cp -R storm-0.7.1 /usr/local/
# vi ~/.bashrc
--
追加
export STORM_HOME=/usr/local/storm-0.7.1
export PATH=$PATH:$STORM_HOME/bin
--
============================
到此为止,storm安装完毕。
二. 本地运行测试程序storm-start
按照https://github.com/nathanmarz/storm-starter,执行这个程序需要用lein,这里介绍的方法用eclipse代替lein的作用。
1) 安装twitter4j
# mkdir twitter4j
# cd twitter4j
# wget http://twitter4j.org/en/twitter4j-2.2.5.zip # unzip twitter4j-2.2.5.zip
2) 追加源文件storm-start/src/jvm/storm
使用eclipse建立java project。追加twitter4j和storm的jar文件。
File-> New -> Java Project ->随便取个名字-> Next -> Libraries -> add External JARs...-> 追加twitter4j和storm的jar文件(/path/to/twitter4j/lib/*.jar和/path/to/storm/lib/*.jar和/path/to/storm/storm-{version}.jar)->
Finsh
导入storm-start
File -> Import -> General -> File System -> Next -> Browse(From directory) -> /path/to/storm-start/src/jvm/storm -> Browse(Info floder)
-> xxx -> src -> OK -> “storm” 和 “Create top-level folder”前打勾 -> Finish
完成之后如图:
3) 追加源文件storm-start/multilang/resources×(python 文件word count用)
File -> Import -> General -> File System -> Next -> Browse(From directory) -> /path/to/storm-start/multilang/resources -> Browse(Info floder)
-> xxx -> OK -> check “resources” and “Create top-level folder” -> Finish
2个源文件都追加好之后,eclipse左边显示如下图:
4) JAR export
File -> Export -> JAR -> JAR file -> 取消 “.classpath” ,“.project” 和 “<.settings” ->的勾 browse -> path/to/export/name.jar -> Finish (忽视 warnings)
4. 执行刚才编译的文件。
# storm jar StormStarter.jar storm.starter.ExclamationTopology
如果出现类似下面的文字,说明运行成功!
....
11367 [Thread-25] INFO
backtype.storm.daemon.task - Emitting: class storm.starter.ExclamationTopology$ExclamationBolt source: 2:3, stream: 1, id: {}, [golda!!!]
....
参考链接:https://github.com/nathanmarz
http://tjun.jp/blog/category/storm/
相关文章推荐
- centos的twitter storm安装和storm-start的本地运行
- twitter storm安装和storm-start的本地运行
- twitter storm安装和storm-start的本地运行
- ubuntu 安装本地版storm并运行WordCount
- centos linux 6.5 yum 从本地光盘安装system-config-kickstart
- CentOS下Storm 1.0.0集群安装详解
- centos 安装 使用本地图像文件来安装网络安装
- 3、单机运行环境搭建之 --CentOS-6.5安装配置Tengine
- RedHat Enterprise (Centos) kickstart 自动安装Linux
- thinkphp3.2.3版本在windows本地apache环境运行正常,上传到centos服务器apache环境中出现:thinkphp 上传根目录不存在!请尝试手动创建:uploads/
- CentOS 安装运行 mongoDB docker镜像
- 在CentOS上安装MongoDB并永久运行
- CentOS下安装使用start-stop-daemon
- centos5.8本地安装yum资源,安装软件包
- Kickstart实现CentOS 6.3的网络自动安装
- Centos5.5 X86_64实战KickStart自动化安装
- CentOS使用YUM安装php运行环境,包含php,php-fpm,nginx,mysql
- pxe+kickstart无人值守自动安装CentOS6.5
- CentOS6.2下一步一步源代码安装OpenStack(五)nova配置、运行、测试
- 【技巧】centos6.5_yum本地安装mysql