running beyond physical memory limits

6/07/19 14:10:30 INFO mapreduce.Job: Task Id : attempt_1459504722049_479821_m_000041_2, Status : FAILED

Container [pid=167394,containerID=container_1459504722049_479821_01_000180] is running beyond physical memory limits. Current usage: 1.1 GB of 1 GB physical memory used; 3.3 GB of 5 GB virtual memory used. Killing container.

Dump of the process-tree for container_1459504722049_479821_01_000180 :

|- PID PPID PGRPID SESSID CMD_NAME USER_MODE_TIME(MILLIS) SYSTEM_TIME(MILLIS) VMEM_USAGE(BYTES) RSSMEM_USAGE(PAGES) FULL_CMD_LINE

|- 167394 167381 167394 167394 (bash) 0 0 108654592 301 /bin/bash -c /usr/local/webserver/jdk//bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx2048m -Djava.io.tmpdir=/data11/server/hadoop/yarn/local/usercache/ad/appcache/application_1459504722049_479821/container_1459504722049_479821_01_000180/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/data1/logs/hadoop/yarn/applogs/application_1459504722049_479821/container_1459504722049_479821_01_000180 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA org.apache.hadoop.mapred.YarnChild 10.1.33.89 10669 attempt_1459504722049_479821_m_000041_2 180 1>/data1/logs/hadoop/yarn/applogs/application_1459504722049_479821/container_1459504722049_479821_01_000180/stdout 2>/data1/logs/hadoop/yarn/applogs/application_1459504722049_479821/container_1459504722049_479821_01_000180/stderr

加上这两个参数就可以跑了

-D mapreduce.map.memory.mb=4096 \

-D mapreduce.map.java.opts=-Xmx1800M \

参考于：

http://stackoverflow.com/questions/21005643/container-is-running-beyond-memory-limits

http://www.ericlin.me/application-log-shows-container-is-running-beyond-physical-memory-limits-error