oozie框架案例之shellAction

oozie配置Shell脚本调度

MapReduce 

Hive

1、拷贝模版

$ cp -r examples/apps/shell/ oozie-apps

2、=========修改job.properties============

The corresponding job properties file used to submit Oozie job could be as follows:

oozie.wf.application.path=hdfs://localhost:8020/user/kamrul/workflows/script#Execute is expected to be in the Workflow directory.
#Shell Script to run
EXEC=script.sh
#CPP executable. Executable should be binary compatible to the compute node OS.
#EXEC=hello
#Perl script
#EXEC=script.pl
jobTracker=localhost:8021
nameNode=hdfs://localhost:8020
queueName=default

nameNode=hdfs://[hostname]:8020

jobTracker=[hostname]:8032

queueName=default

examplesRoot=oozie-apps

oozie.wf.application.path=${nameNode}/user/admin/${examplesRoot}/shell

EXEC=mem.sh

3、修改==============workflow.xml=============

How to run any shell script or perl script or CPP executable

<workflow-app xmlns='uri:oozie:workflow:0.3' name='shell-wf'>
<start to='shell1' />
<action name='shell1'>
<shell xmlns="uri:oozie:shell-action:0.1">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<configuration>
<property>
<name>mapred.job.queue.name</name>
<value>${queueName}</value>
</property>
</configuration>
<exec>${EXEC}</exec>
<argument>A</argument>
<argument>B</argument>
<file>${EXEC}#${EXEC}</file> <!--Copy the executable to compute node's current working directory -->
</shell>
<ok to="end" />
<error to="fail" />
</action>
<kill name="fail">
<message>Script failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>
</kill>
<end name='end' />
</workflow-app>

<start to="shell-node"/>

    <action name="shell-node">

        <shell xmlns="uri:oozie:shell-action:0.2">

            <job-tracker>${jobTracker}</job-tracker>

            <name-node>${nameNode}</name-node>

            <configuration>

                <property>

                    <name>mapred.job.queue.name</name>

                    <value>${queueName}</value>

                </property>

            </configuration>

            <exec>${EXEC}</exec>

            <file>${nameNode}/user/user01/${examplesRoot}/shell/${EXEC}#${EXEC}</file>

        </shell>

        <ok to="end"/>

        <error to="fail"/>

    </action>   

    <kill name="fail">

        <message>Shell action failed, error message[${wf:errorMessage(wf:lastErrorNode())}]</message>

    </kill>   

    <end name="end"/>

4、创建测试脚本

#!/bin/sh

/bin/date -R >> /opt/modules/cdh/oozie-4.0.0-cdh5.3.6/oozie-apps/shell/

1.log

/usr/bin/free -m >> /opt/modules/cdh/oozie-4.0.0-cdh5.3.6/oozie-apps/sh

ell/1.log

/bin/df -l >> /opt/modules/cdh/oozie-4.0.0-cdh5.3.6/oozie-apps/shell/1.

log

echo ------------------- >> /opt/modules/cdh/oozie-4.0.0-cdh5.3.6/oozie

-apps/shell/1.log

5.上传到HDFS对应目录

$ bin/hdfs dfs -put /opt/modules/oozie-4.0.0-cdh5.3.6/oozie-apps/shell /user/user01/oozie-apps/

6、执行：

$ bin/oozie job -oozie http://[hostname]:11000/oozie -config oozie-apps/shell/job.properties -run

$ bin/oozie job -oozie http://[hostname]:11000/oozie -config oozie-apps/shell/job.properties -run

以上是配置oozie的手动提交 --workflow