[Hadoop][2017-06-27] lzo.lzocodec not found
2017-06-27 15:46
639 查看
在编译 Demo 时报错 hadoop jar WordCount.jar WordCount /input /output
注意:/input /output /表示绝对路径, 不加为相对路径
此问题, 解决之道
hadoop-lzo-0.4.20-SNAPSHOT.jar 此文件, 需要放入hadoop所有节点上, NameNode & DataNode
菜鸟基本的错误,google根本不到。。。
原始报错信息如下:
17/06/27 15:18:01 INFO mapreduce.Job: Task Id : attempt_1498546574227_0003_m_000001_1001, Status : FAILED Error: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
at org.apache.hadoop.mapred.JobConf.getInputFormat(JobConf.java:689)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
... 11 more
Caused by: java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:139)
at org.apache.hadoop.io.compress.CompressionCodecFactory.(CompressionCodecFactory.java:179)
at org.apache.hadoop.mapred.TextInputFormat.configure(TextInputFormat.java:45)
... 16 more
Caused by: java.lang.ClassNotFoundException: Class com.hadoop.compression.lzo.LzoCodec not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:132)
... 18 more
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
注意:/input /output /表示绝对路径, 不加为相对路径
此问题, 解决之道
hadoop-lzo-0.4.20-SNAPSHOT.jar 此文件, 需要放入hadoop所有节点上, NameNode & DataNode
菜鸟基本的错误,google根本不到。。。
原始报错信息如下:
17/06/27 15:18:01 INFO mapreduce.Job: Task Id : attempt_1498546574227_0003_m_000001_1001, Status : FAILED Error: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:112)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:78)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136)
at org.apache.hadoop.mapred.JobConf.getInputFormat(JobConf.java:689)
at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.(MapTask.java:169)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:432)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:109)
... 11 more
Caused by: java.lang.IllegalArgumentException: Compression codec com.hadoop.compression.lzo.LzoCodec not found.
at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:139)
at org.apache.hadoop.io.compress.CompressionCodecFactory.(CompressionCodecFactory.java:179)
at org.apache.hadoop.mapred.TextInputFormat.configure(TextInputFormat.java:45)
... 16 more
Caused by: java.lang.ClassNotFoundException: Class com.hadoop.compression.lzo.LzoCodec not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2101)
at org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:132)
... 18 more
Container killed by the ApplicationMaster.
Container killed on request. Exit code is 143
Container exited with a non-zero exit code 143
相关文章推荐
- spark取得lzo压缩文件报错 java.lang.ClassNotFoundException: Class com.hadoop.compression.lzo.LzoCodec
- hadoop中的LzoCodec和LzopCodec
- hadoop运行过程中报错:ERROR lzo.LzoCodec: Cannot load native-lzo without native-hadoop
- spark取得lzo压缩文件报错 java.lang.ClassNotFoundException: Class com.hadoop.compression.lzo.LzoCodec
- emr+hadoop2.4+spark1.2 class not found com.hadoop.compression.lzo.LzoCodec
- centos6 上用eclipse调试hadoop程序报org.apache.hadoop.io.compress.SnappyCodec not found错误解决方法
- spark Compression codec com.hadoop.compression.lzo.LzoCodec not found
- spark异常 Compression codec com.hadoop.compression.lzo.LzoCodec not found
- hadoop streaming python 处理 lzo 文件遇到的问题
- hadoop深入研究-(八)——codec
- hadoop-lzo 安装教程
- org.apache.hadoop.hive.contrib.serde2.RegexSerDe not found
- 【Hadoop/Hbase】centos上安装并设置Snappy/LZO压缩方式
- hadoop平台gz、lzo压缩对比
- 快速为CDH版本HADOOP安装配置Lzo和Snappy压缩
- hadoop环境搭建遇到的问题:-bash: jps: command not found
- hadoop 2.0 lzo 问题
- Hadoop | SequenceFile doesn't work with GzipCodec without native-hadoop code 异常解决
- hadoop深入研究-(八)——codec
- hadoop深入研究:(八)——codec