hadoop 安装出现的几种异常的处理方法
2012-12-06 03:31
495 查看
hadoop无法正常启动(1)执行 $ bin/hadoop start-all.sh之后,无法启动.异常一 Exception in thread "main" java.lang.IllegalArgumentException: Invalid URI for NameNode address (check fs.defaultFS): file:/// has no authority.localhost: at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:214)localhost: at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:135)localhost: at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:119)localhost: at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:481)
解决方法:此时是没有配置conf/mapred-site.xml的缘故. 在0.21.0版本上是配置mapred-site.xml,在之前的版本是配置core-site.xml,0.20.2版本中配置mapred-site.xml无效,只能配置core-site.xml文件<configuration> <property> <name>fs.default.name</name> <value>hdfs://localhost:9000</value> </property> <property> <name>mapred.job.tracker</name> <value>hdfs://localhost:9001</value> </property> <property> <name>dfs.replication</name> <value>1</value> </property></configuration> hadoop无法正常启动(2)异常二、starting namenode, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-namenode-aist.outlocalhost: starting datanode, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-datanode-aist.outlocalhost: starting secondarynamenode, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-secondarynamenode-aist.outlocalhost: Exception in thread "main" java.lang.NullPointerExceptionlocalhost: at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:134)localhost: at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:156)localhost: at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:160)localhost: at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:131)localhost: at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:115)localhost: at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:469)starting jobtracker, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-jobtracker-aist.outlocalhost: starting tasktracker, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-tasktracker-aist.out 解决方法:此时是没有配置conf/mapred-site.xml的缘故. 在0.21.0版本上是配置mapred-site.xml,在之前的版本是配置core-site.xml , 0.20.2版本中配置mapred-site.xml无效,只能配置core-site.xml文件<configuration> <property> <name>fs.default.name</name> <value>hdfs://localhost:9000</value> </property> <property> <name>mapred.job.tracker</name> <value>hdfs://localhost:9001</value> </property> <property> <name>dfs.replication</name> <value>1</value> </property></configuration> hadoop无法正常启动(3)异常三、starting namenode, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-namenode-aist.outlocalhost: starting datanode, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-datanode-aist.outlocalhost: Error: JAVA_HOME is not set.localhost: starting secondarynamenode, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-secondarynamenode-aist.outlocalhost: Error: JAVA_HOME is not set.starting jobtracker, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-jobtracker-aist.outlocalhost: starting tasktracker, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-tasktracker-aist.outlocalhost: Error: JAVA_HOME is not set. 解决方法:请在$hadoop/conf/hadoop-env.sh文件中配置JDK的环境变量JAVA_HOME=/home/xixitie/jdkCLASSPATH=$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jarexport JAVA_HOME CLASSPATH
hadoop无法正常启动(4)异常四:mapred-site.xml配置中使用hdfs://localhost:9001,而不使用localhost:9001的配置异常信息如下:11/04/20 23:33:25 INFO security.Groups: Group mapping impl=org.apache.hadoop.sec urity.ShellBasedUnixGroupsMapping; cacheTimeout=30000011/04/20 23:33:25 WARN fs.FileSystem: "localhost:9000" is a deprecated filesyste m name. Use "hdfs://localhost:9000/" instead.11/04/20 23:33:25 WARN conf.Configuration: mapred.task.id is deprecated. Instead , use mapreduce.task.attempt.id11/04/20 23:33:25 WARN fs.FileSystem: "localhost:9000" is a deprecated filesyste m name. Use "hdfs://localhost:9000/" instead.11/04/20 23:33:25 WARN fs.FileSystem: "localhost:9000" is a deprecated filesyste m name. Use "hdfs://localhost:9000/" instead. 解决方法:mapred-site.xml配置中使用hdfs://localhost:9000,而不使用localhost:9000的配置 <property> <name>fs.default.name</name> <value>hdfs://localhost:9000</value> </property> <property> <name>mapred.job.tracker</name> <value>hdfs://localhost:9001</value> </property> hadoop无法正常启动(5)异常五、no namenode to stop 问题的解决:异常信息如下:
11/04/20 21:48:50 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 0 time(s).11/04/20 21:48:51 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 1 time(s).11/04/20 21:48:52 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 2 time(s).11/04/20 21:48:53 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 3 time(s).11/04/20 21:48:54 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 4 time(s).11/04/20 21:48:55 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 5 time(s).11/04/20 21:48:56 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 6 time(s).11/04/20 21:48:57 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 7 time(s).11/04/20 21:48:58 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 8 time(s). 解决方法:这个问题是由namenode没有启动起来引起的,为什么no namenode to stop,可能之前的一些数据对namenode有影响,你需要执行:$ bin/hadoop namenode -format然后$bin/hadoop start-all.sh
hadoop无法正常启动(6)异常五、no datanode to stop 问题的解决:有时数据结构出现问题会产生无法启动datanode的问题。然后用 hadoop namenode -format 重新格式化后仍然无效,/tmp中的文件并没有清楚。其实还需要清除/tmp/hadoop*里的文件。执行步骤: 一、先删除hadoop:///tmp hadoop fs -rmr /tmp 二、停止 hadoop stop-all.sh 三、删除/tmp/hadoop* rm -rf /tmp/hadoop* 四、格式化hadoop hadoop namenode -format 五、启动hadoop start-all.sh之后即可解决这个datanode没法启动的问题本文出自 “左罗CTO” 博客,请务必保留此出处http://zorro.blog.51cto.com/2139862/1079979
解决方法:此时是没有配置conf/mapred-site.xml的缘故. 在0.21.0版本上是配置mapred-site.xml,在之前的版本是配置core-site.xml,0.20.2版本中配置mapred-site.xml无效,只能配置core-site.xml文件<configuration> <property> <name>fs.default.name</name> <value>hdfs://localhost:9000</value> </property> <property> <name>mapred.job.tracker</name> <value>hdfs://localhost:9001</value> </property> <property> <name>dfs.replication</name> <value>1</value> </property></configuration> hadoop无法正常启动(2)异常二、starting namenode, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-namenode-aist.outlocalhost: starting datanode, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-datanode-aist.outlocalhost: starting secondarynamenode, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-secondarynamenode-aist.outlocalhost: Exception in thread "main" java.lang.NullPointerExceptionlocalhost: at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:134)localhost: at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:156)localhost: at org.apache.hadoop.hdfs.server.namenode.NameNode.getAddress(NameNode.java:160)localhost: at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.initialize(SecondaryNameNode.java:131)localhost: at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.<init>(SecondaryNameNode.java:115)localhost: at org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode.main(SecondaryNameNode.java:469)starting jobtracker, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-jobtracker-aist.outlocalhost: starting tasktracker, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-tasktracker-aist.out 解决方法:此时是没有配置conf/mapred-site.xml的缘故. 在0.21.0版本上是配置mapred-site.xml,在之前的版本是配置core-site.xml , 0.20.2版本中配置mapred-site.xml无效,只能配置core-site.xml文件<configuration> <property> <name>fs.default.name</name> <value>hdfs://localhost:9000</value> </property> <property> <name>mapred.job.tracker</name> <value>hdfs://localhost:9001</value> </property> <property> <name>dfs.replication</name> <value>1</value> </property></configuration> hadoop无法正常启动(3)异常三、starting namenode, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-namenode-aist.outlocalhost: starting datanode, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-datanode-aist.outlocalhost: Error: JAVA_HOME is not set.localhost: starting secondarynamenode, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-secondarynamenode-aist.outlocalhost: Error: JAVA_HOME is not set.starting jobtracker, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-jobtracker-aist.outlocalhost: starting tasktracker, logging to /home/xixitie/hadoop/bin/../logs/hadoop-root-tasktracker-aist.outlocalhost: Error: JAVA_HOME is not set. 解决方法:请在$hadoop/conf/hadoop-env.sh文件中配置JDK的环境变量JAVA_HOME=/home/xixitie/jdkCLASSPATH=$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jarexport JAVA_HOME CLASSPATH
hadoop无法正常启动(4)异常四:mapred-site.xml配置中使用hdfs://localhost:9001,而不使用localhost:9001的配置异常信息如下:11/04/20 23:33:25 INFO security.Groups: Group mapping impl=org.apache.hadoop.sec urity.ShellBasedUnixGroupsMapping; cacheTimeout=30000011/04/20 23:33:25 WARN fs.FileSystem: "localhost:9000" is a deprecated filesyste m name. Use "hdfs://localhost:9000/" instead.11/04/20 23:33:25 WARN conf.Configuration: mapred.task.id is deprecated. Instead , use mapreduce.task.attempt.id11/04/20 23:33:25 WARN fs.FileSystem: "localhost:9000" is a deprecated filesyste m name. Use "hdfs://localhost:9000/" instead.11/04/20 23:33:25 WARN fs.FileSystem: "localhost:9000" is a deprecated filesyste m name. Use "hdfs://localhost:9000/" instead. 解决方法:mapred-site.xml配置中使用hdfs://localhost:9000,而不使用localhost:9000的配置 <property> <name>fs.default.name</name> <value>hdfs://localhost:9000</value> </property> <property> <name>mapred.job.tracker</name> <value>hdfs://localhost:9001</value> </property> hadoop无法正常启动(5)异常五、no namenode to stop 问题的解决:异常信息如下:
11/04/20 21:48:50 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 0 time(s).11/04/20 21:48:51 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 1 time(s).11/04/20 21:48:52 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 2 time(s).11/04/20 21:48:53 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 3 time(s).11/04/20 21:48:54 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 4 time(s).11/04/20 21:48:55 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 5 time(s).11/04/20 21:48:56 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 6 time(s).11/04/20 21:48:57 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 7 time(s).11/04/20 21:48:58 INFO ipc.Client: Retrying connect to server: localhost/127.0.0 .1:9000. Already tried 8 time(s). 解决方法:这个问题是由namenode没有启动起来引起的,为什么no namenode to stop,可能之前的一些数据对namenode有影响,你需要执行:$ bin/hadoop namenode -format然后$bin/hadoop start-all.sh
hadoop无法正常启动(6)异常五、no datanode to stop 问题的解决:有时数据结构出现问题会产生无法启动datanode的问题。然后用 hadoop namenode -format 重新格式化后仍然无效,/tmp中的文件并没有清楚。其实还需要清除/tmp/hadoop*里的文件。执行步骤: 一、先删除hadoop:///tmp hadoop fs -rmr /tmp 二、停止 hadoop stop-all.sh 三、删除/tmp/hadoop* rm -rf /tmp/hadoop* 四、格式化hadoop hadoop namenode -format 五、启动hadoop start-all.sh之后即可解决这个datanode没法启动的问题本文出自 “左罗CTO” 博客,请务必保留此出处http://zorro.blog.51cto.com/2139862/1079979
相关文章推荐
- hadoop 安装出现的几种异常的处理方法,hadoop无法启动,no namenode to stop问题的解决方法,no datanode
- hadoop 安装出现的几种异常的处理方法,hadoop无法启动,no namenode to stop问题的解决方法,no datanode
- hadoop 安装出现的几种异常的处理方法
- 当调用方法时出现多个异常类时,对应的两种处理方案
- asp.net捕获全局未处理异常的几种方法[转]
- 如果reporting server出现“无法安装打印控件”的处理方法。
- mac下使用scrapy时出现的raise DistributionNotFound(req)异常的处理方法
- SQLserver2000 企业版 出现"进程51发生了严重的异常"错误的处理方法
- CVM下如出现异常情况的处理方法
- [转]关于安装hadoop中出现的的 $HADOOP_HOME is deprecated 的解决方法
- 调试出现LoaderLock异常的几种解决方法
- 关于vs2013 安装后出现"Cannot find one of more components. Please reinstall the application"处理方法
- 安装Sphinx 2.0.4时出现的以下几个错误处理方法 WARNING: compat_sphinxql_magics=1 is deprecated;
- Intellij IDEA中Spring配置aop增强事务 出现BeanCreationException异常处理方法
- 项目开发时,出现异常,但控制台没有打印错误信息处理方法
- 使用MediaPlayer出现The surface has been released 异常处理方法,附代码
- arcgis安装不成功的几种处理方法
- 异常:安装完Hadoop之后,命令行输入hadoop却找不到命令的解决方法
- hadoop常见异常及处理方法
- SQL2000 企业版 出现“进程51发生了严重的异常”错误的处理方法