oozie配置sharelib,java.lang.NoClassDefFoundError: org/apache/pig/Main

作为笔记：

sharelib的配置

在用oozie调用pig时，用到sharelib，当时是直接运行的，然后就报错了.

2013-07-03 20:44:48,504 WARN PigActionExecutor:542 - USER[root] GROUP[-] TOKEN[] APP[pig-wf] JOB[0000004-130703183035408-oozie-root-W] ACTION[0000004-130703183035408-oozie-root-W@pig-node]
Launcher exception: org/apache/pig/Main

java.lang.NoClassDefFoundError: org/apache/pig/Main

at org.apache.oozie.action.hadoop.PigMain.runPigJob(PigMain.java:323)

at org.apache.oozie.action.hadoop.PigMain.run(PigMain.java:218)

at org.apache.oozie.action.hadoop.LauncherMain.run(LauncherMain.java:37)

at org.apache.oozie.action.hadoop.PigMain.main(PigMain.java:76)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

at java.lang.reflect.Method.invoke(Method.java:597)

at org.apache.oozie.action.hadoop.LauncherMapper.map(LauncherMapper.java:467)

at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)

at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:396)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)

at org.apache.hadoop.mapred.Child.main(Child.java:249)

Caused by: java.lang.ClassNotFoundException: org.apache.pig.Main

at java.net.URLClassLoader$1.run(URLClassLoader.java:202)

at java.security.AccessController.doPrivileged(Native Method)

at java.net.URLClassLoader.findClass(URLClassLoader.java:190)

at java.lang.ClassLoader.loadClass(ClassLoader.java:306)

at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)

at java.lang.ClassLoader.loadClass(ClassLoader.java:247)

... 17 more

最后在网上各种扒问题，发现是需要sharelib，action执行时，会对sharelib进行依赖，而sharelib要放到hdfs上。

oozie提供一个sharelib：oozie-sharelib-**-incubating.tar.gz。解压即可。

然后需要把sharelib放到hdfs上，hdfs的sharelib路径发现也是个问题。

这个需要在oozie安装目录下conf/oozie-default.xml中查找：

<property>

<name>oozie.service.WorkflowAppService.system.libpath</name>

<value>/user/${user.name}/share/lib</value>

<description>

System library path to use for workflow applications.

This path is added to workflow application if their job properties sets

the property 'oozie.use.system.libpath' to true.

</description>

</property>

“/user/${user.name}/share/lib”就是我们需要放sharelib的路径。