您的位置:首页 > 运维架构

[Hadoop][2017-06-27] lzo.lzocodec not found

2017-06-27 15:46 639 查看
在编译 Demo 时报错  hadoop jar WordCount.jar WordCount /input /output 

注意:/input /output    /表示绝对路径, 不加为相对路径 

此问题, 解决之道

hadoop-lzo-0.4.20-SNAPSHOT.jar   此文件, 需要放入hadoop所有节点上, NameNode & DataNode

菜鸟基本的错误,google根本不到。。。

原始报错信息如下:

17/06/27 15:18:01 INFO mapreduce.Job: Task Id : attempt_1498546574227_0003_m_000001_1001, Status : FAILED Error: java.lang.RuntimeException: Error in configuring object 

at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112) 

at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78) 

at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) 

at org.apache.hadoop.mapred.JobConf.getInputFormat(JobConf.java:689) 

at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.(MapTask.java:169) 

at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432) 

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) 

at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) 

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698) 

at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: java.lang.reflect.InvocationTargetException 

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) 

at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) 

at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) 

at java.lang.reflect.Method.invoke(Method.java:498) 

at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109) 

... 11 more 

Caused by: java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec not found. 

at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:139) 

at org.apache.hadoop.io.compress.CompressionCodecFactory.(CompressionCodecFactory.java:179) 

at org.apache.hadoop.mapred.TextInputFormat.configure(TextInputFormat.java:45)

 ... 16 more 

Caused by: java.lang.ClassNotFoundException: Class com.hadoop.compression.lzo.LzoCodec not found 

at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101) 

at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:132) 

... 18 more 

Container killed by the ApplicationMaster. 

Container killed on request. Exit code is 143 

Container exited with a non-zero exit code 143
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签:  hadoop lzo