hadoop常见错误

用eclipse做客户端的时候，会遇到内存溢出的问题

13/06/25 19:15:21 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

13/06/25 19:15:21 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.

13/06/25 19:15:21 WARN mapred.JobClient: No job jar file set. User classes may not be found. See JobConf(Class) or JobConf#setJar(String).

13/06/25 19:15:21 INFO input.FileInputFormat: Total input paths to process : 2

13/06/25 19:15:21 WARN snappy.LoadSnappy: Snappy native library not loaded

13/06/25 19:15:22 INFO mapred.JobClient: Running job: job_local_0001

13/06/25 19:15:23 INFO mapred.Task: Using ResourceCalculatorPlugin : null

13/06/25 19:15:23 INFO mapred.MapTask: io.sort.mb = 100

13/06/25 19:15:23 WARN mapred.LocalJobRunner: job_local_0001

java.lang.OutOfMemoryError: Java heap space

at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:949)

at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:674)

at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:756)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:214)

13/06/25 19:15:23 INFO mapred.JobClient: map 0% reduce 0%

13/06/25 19:15:23 INFO mapred.JobClient: Job complete: job_local_0001

13/06/25 19:15:23 INFO mapred.JobClient: Counters: 0

解决方案是将eclipse运行时的堆栈调大

点击项目右键-->Run As-->Run Configurations，在配置运行参数，点Arguments，添加

-Xms512m -Xmx1024m -XX:MaxPermSize=256m