【原】centos6.5下cdh4.6 Oozie安装
2014-05-09 09:33
295 查看
0、oozie只需安装在一台服务器上,这里选择在namenode上来安装;安装用户为cloud-user
1、安装Oozie包:
sudo yum install -y oozie oozie-client
2、对于yarn来说,需要修改/etc/oozie/conf/oozie-env.sh中:
CATALINA_BASE=/usr/lib/oozie/oozie-server
3、配置mysql:
yum install -y mysql-server
创建数据库:
mysql -u root -p
[align=left] mysql> create database oozie;[/align]
[align=left] mysql> grant all privileges on oozie.* to 'oozie'@'localhost' identified by 'oozie';[/align]
[align=left] mysql> grant all privileges on oozie.* to 'oozie'@'%' identified by 'oozie';[/align]
[align=left] mysql> exit[/align]
4、配置oozie-site.xml:
[align=left]5、拷贝JDBC Driver:[/align]
[align=left] sudo cp /usr/share/java/mysql-connector-java.jar /var/lib/oozie[/align]
[align=left] sudo cp /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar /var/lib/oozie[/align]
[align=left] sudo cp /usr/lib/hadoop/lib/hadoop-lzo.jar /var/lib/oozie[/align]
[align=left]6、创建oozie 数据库scheme,这一步骤需要root来执行:[/align]
[align=left] su - [/align]
[align=left] /usr/lib/oozie/bin/ooziedb.sh create -sqlfile oozie-create.sql[/align]
[align=left] /usr/lib/oozie/bin/ooziedb.sh create -run [/align]
[align=left]7、使能web console:[/align]
[align=left] wget http://archive.cloudera.com/gplextras/misc/ext-2.2.zip[/align]
[align=left] sudo mv ext-2.2.zip /var/lib/oozie/[/align]
[align=left] cd /var/lib/oozie/ && sudo unzip ext-2.2.zip[/align]
[align=left] sudo chown -R oozie:oozie /var/lib/oozie/[/align]
[align=left]8、安装Oozie ShareLib:[/align]
[align=left] sudo -u hdfs hadoop fs -mkdir /user/oozie[/align]
[align=left] sudo -u hdfs hadoop fs -chown oozie:oozie /user/oozie[/align]
[align=left] mkdir /tmp/ooziesharelib[/align]
[align=left] cd /tmp/ooziesharelib[/align]
[align=left] tar xzf /usr/lib/oozie/oozie-sharelib-yarn.tar.gz[/align]
[align=left] sudo -u oozie hadoop fs -put share /user/oozie/share[/align]
[align=left]9、启动服务:[/align]
[align=left] sudo service oozie start[/align]
[align=left]10、测试map-reduce:[/align]
[align=left] 添加/etc/hadoop/conf/core-site.xml[/align]
cp /usr/share/doc/oozie-3.3.2+100/oozie-examples.tar.gz .
tar -zxvf oozie-examples.tar.gz
修改example/apps/map-reduce/job.properties
nameNode=hdfs://10.0.0.2:8020
jobTracker=10.0.0.2:8032 端口改为/etc/hadoop/conf/yarn-site.xml中 'yarn.resourcemanager.address ' 项的值
hadoop fs -put examples
oozie job -oozie http://10.0.0.3:11000/oozie -config ./examples/apps/no-op/job.properties -run
Job ID : 0000000-140508214215146-oozie-oozi-W
通过oozie job -oozie http://10.0.0.3:11000/oozie -log 0000000-140508214215146-oozie-oozi-W查看log
通过oozie job -oozie http://10.0.0.3:11000/oozie -info 0000000-140508214215146-oozie-oozi-W查看信息
通过http://namenode:11000/oozie查看web console
1、安装Oozie包:
sudo yum install -y oozie oozie-client
2、对于yarn来说,需要修改/etc/oozie/conf/oozie-env.sh中:
CATALINA_BASE=/usr/lib/oozie/oozie-server
3、配置mysql:
yum install -y mysql-server
创建数据库:
mysql -u root -p
[align=left] mysql> create database oozie;[/align]
[align=left] mysql> grant all privileges on oozie.* to 'oozie'@'localhost' identified by 'oozie';[/align]
[align=left] mysql> grant all privileges on oozie.* to 'oozie'@'%' identified by 'oozie';[/align]
[align=left] mysql> exit[/align]
4、配置oozie-site.xml:
<property> <name>oozie.service.JPAService.jdbc.driver</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>oozie.service.JPAService.jdbc.url</name> <value>jdbc:mysql://10.0.0.2:3306/oozie</value> </property> <property> <name>oozie.service.JPAService.jdbc.username</name> <value>oozie</value> </property> <property> <name>oozie.service.JPAService.jdbc.password</name> <value>oozie</value> </property> <property> <name>oozie.action.mapreduce.uber.jar.enable</name> <value>true</value> </property> <property> <name>oozie.service.HadoopAccessorService.supported.filesystems</name> <value>hdfs,viewfs</value> </property> <property> <name>oozie.service.ProxyUserService.proxyuser.cloud-user.hosts</name> <value>*</value> </property> <property> <name>oozie.service.ProxyUserService.proxyuser.cloud-user.groups</name> <value>*</value> </property>
[align=left]5、拷贝JDBC Driver:[/align]
[align=left] sudo cp /usr/share/java/mysql-connector-java.jar /var/lib/oozie[/align]
[align=left] sudo cp /usr/lib/hadoop/lib/hadoop-lzo-cdh4-0.4.15-gplextras.jar /var/lib/oozie[/align]
[align=left] sudo cp /usr/lib/hadoop/lib/hadoop-lzo.jar /var/lib/oozie[/align]
[align=left]6、创建oozie 数据库scheme,这一步骤需要root来执行:[/align]
[align=left] su - [/align]
[align=left] /usr/lib/oozie/bin/ooziedb.sh create -sqlfile oozie-create.sql[/align]
[align=left] /usr/lib/oozie/bin/ooziedb.sh create -run [/align]
[align=left]7、使能web console:[/align]
[align=left] wget http://archive.cloudera.com/gplextras/misc/ext-2.2.zip[/align]
[align=left] sudo mv ext-2.2.zip /var/lib/oozie/[/align]
[align=left] cd /var/lib/oozie/ && sudo unzip ext-2.2.zip[/align]
[align=left] sudo chown -R oozie:oozie /var/lib/oozie/[/align]
[align=left]8、安装Oozie ShareLib:[/align]
[align=left] sudo -u hdfs hadoop fs -mkdir /user/oozie[/align]
[align=left] sudo -u hdfs hadoop fs -chown oozie:oozie /user/oozie[/align]
[align=left] mkdir /tmp/ooziesharelib[/align]
[align=left] cd /tmp/ooziesharelib[/align]
[align=left] tar xzf /usr/lib/oozie/oozie-sharelib-yarn.tar.gz[/align]
[align=left] sudo -u oozie hadoop fs -put share /user/oozie/share[/align]
[align=left]9、启动服务:[/align]
[align=left] sudo service oozie start[/align]
[align=left]10、测试map-reduce:[/align]
[align=left] 添加/etc/hadoop/conf/core-site.xml[/align]
<property> <name>hadoop.proxyuser.oozie.hosts</name> <value>10.0.0.2</value> </property> <property> <name>hadoop.proxyuser.oozie.groups</name> <value>*</value> </property>
cp /usr/share/doc/oozie-3.3.2+100/oozie-examples.tar.gz .
tar -zxvf oozie-examples.tar.gz
修改example/apps/map-reduce/job.properties
nameNode=hdfs://10.0.0.2:8020
jobTracker=10.0.0.2:8032 端口改为/etc/hadoop/conf/yarn-site.xml中 'yarn.resourcemanager.address ' 项的值
hadoop fs -put examples
oozie job -oozie http://10.0.0.3:11000/oozie -config ./examples/apps/no-op/job.properties -run
Job ID : 0000000-140508214215146-oozie-oozi-W
通过oozie job -oozie http://10.0.0.3:11000/oozie -log 0000000-140508214215146-oozie-oozi-W查看log
通过oozie job -oozie http://10.0.0.3:11000/oozie -info 0000000-140508214215146-oozie-oozi-W查看信息
通过http://namenode:11000/oozie查看web console
相关文章推荐
- 【原】centos6.5下hadoop cdh4.6 安装
- 【原】Centos6.5下cdh4.6 hive安装部署
- Hbase学习1_CentOS6.5下Hbase1.1.2安装
- CentOS 6.5下安装NumPy、SciPy、Scikit-Learn
- centos6.5下mysql安装与配置
- Centos6.5下Redis安装、主从配置、卸载教程
- CentOS6.5下的时间同步ntpdate安装
- CentOS release 6.5下安装php-memcached
- centos6.5下安装mysql,远程访问
- CentOS 6.5下Redmine的安装配置
- CentOS6.5下Redis安装与配置
- CentOS6.5下安装ActiveMQ
- CentOS 6.5下安装MySQL 5.6.35
- CentOS6.5下Nginx1.7.4安装记录
- CentOS 6.5下Redis安装记录
- CentOS6.5下RPM方式安装MySQL5.6
- mysql5.7.10 centos 6.5下安装
- CentOS6.5下安装mongodb
- CentOS6.5下安装nginx1.10
- Centos 6.5下安装vsftpd服务器