Mapreduce Error: Type mismatch in key from map
2013-07-22 21:12
399 查看
关于Mapreduce中出现的错误:Type mismatch in key from map: expected **, recieved org.apache.hadoop.io.LongWritable
例如:
13/07/22 02:53:32 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
13/07/22 02:53:32 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/07/22 02:53:32 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
13/07/22 02:53:32 INFO input.FileInputFormat: Total input paths to process : 1
13/07/22 02:53:38 INFO mapred.JobClient: Running job: job_local_0001
13/07/22 02:53:38 INFO input.FileInputFormat: Total input paths to process : 1
13/07/22 02:53:38 INFO mapred.MapTask: io.sort.mb = 100
13/07/22 02:53:39 INFO mapred.JobClient: map 0% reduce 0%
13/07/22 02:53:39 INFO mapred.MapTask: data buffer = 79691776/99614720
13/07/22 02:53:39 INFO mapred.MapTask: record buffer = 262144/327680
13/07/22 02:53:39 WARN mapred.LocalJobRunner: job_local_0001
java.io.IOException: Type mismatch in key from map: expected CoOccurrence$TextPair, recieved org.apache.hadoop.io.LongWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:845)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:541)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:124)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
13/07/22 02:53:40 INFO mapred.JobClient: Job complete: job_local_0001
13/07/22 02:53:40 INFO mapred.JobClient: Counters: 0
出现这个错误的原因:
1、map和reduce中的输入输出格式不对。
2、新旧api混用。你的map()方法 没有按新api写, 结果系统不认为它是一个重载,而是一个新方法,不会被调用。
解决办法:在的map(), reduce()前面加上@Override,并按照新的API来写map(), reduce()。
例如:
13/07/22 02:53:32 INFO jvm.JvmMetrics: Initializing JVM Metrics with processName=JobTracker, sessionId=
13/07/22 02:53:32 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/07/22 02:53:32 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).
13/07/22 02:53:32 INFO input.FileInputFormat: Total input paths to process : 1
13/07/22 02:53:38 INFO mapred.JobClient: Running job: job_local_0001
13/07/22 02:53:38 INFO input.FileInputFormat: Total input paths to process : 1
13/07/22 02:53:38 INFO mapred.MapTask: io.sort.mb = 100
13/07/22 02:53:39 INFO mapred.JobClient: map 0% reduce 0%
13/07/22 02:53:39 INFO mapred.MapTask: data buffer = 79691776/99614720
13/07/22 02:53:39 INFO mapred.MapTask: record buffer = 262144/327680
13/07/22 02:53:39 WARN mapred.LocalJobRunner: job_local_0001
java.io.IOException: Type mismatch in key from map: expected CoOccurrence$TextPair, recieved org.apache.hadoop.io.LongWritable
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:845)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:541)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at org.apache.hadoop.mapreduce.Mapper.map(Mapper.java:124)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
13/07/22 02:53:40 INFO mapred.JobClient: Job complete: job_local_0001
13/07/22 02:53:40 INFO mapred.JobClient: Counters: 0
出现这个错误的原因:
1、map和reduce中的输入输出格式不对。
2、新旧api混用。你的map()方法 没有按新api写, 结果系统不认为它是一个重载,而是一个新方法,不会被调用。
解决办法:在的map(), reduce()前面加上@Override,并按照新的API来写map(), reduce()。
相关文章推荐
- Mapreduce Error: Type mismatch in key from map
- 问题: Type mismatch in key from map: expected org.apache.hadoop.io.Text, recieved org.apache.hadoop.io.LongWritable 解决方案
- hadoop中mapreduce的 Type mismatch in value from map错误
- Type mismatch in key from map: expected org.apache.hadoop.io.Text, received org.apache.hadoop.io.Lon
- java.io.IOException: Type mismatch in key from map: expected
- hadoop中mapreduce的 Type mismatch in value from map错误
- IOException: Type mismatch in key from map的处理
- java.io.IOException: Type mismatch in key from map:解决方法
- java.io.IOException: Type mismatch in key from map: expected org.apache.hadoop.io.Text, recieved org
- Type mismatch in key from map: expected org.apache.hadoop.io.LongWritable, received org.apache.hadoo
- hadoop下 Type mismatch in value from map的问题
- hadoop-1.0.4 Type mismatch in value from map解决方法
- hadoop问题Type mismatch in value from map解决方法
- Hadoop on Mac with IntelliJ IDEA - 9 解决Type mismatch in value from map问题
- hadoop问题Type mismatch in value from map解决方法
- hadoop问题Type mismatch in value from map解决方法
- 【ActiveMQ】管理界面查看消息详情,报错/WEB-INF/tags/form/forEachMapEntry.tag PWC6199: Generated servlet error: The type java.util.Map$Entry cannot be resolved. It is indirectly referenced from required .class files
- Use a Spring InitBinder to Resolve Type Mismatch and Bind Exceptions in POST from Spring Framework M
- NoSuchFieldError: No static field btn_press3 of type I in class Lcom/hyphenate/helpdesk/R$id; or its
- ERROR:duplicate key in table MYsql