Hadoop中碰到的一些错误的解决方法
2013-01-19 01:33
519 查看
Hadoop 0.20.2安装启动后 jps找不到NameNode 解决方案 单机伪分布式
cat hadoop-hadoop-namenode-ubuntu.log:
2013-01-19 00:34:55,812 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.
java.io.IOException: NameNode is not formatted.
at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:317)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:87)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:311)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:292)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:201)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:279)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:956)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965)
2013-01-19 00:34:55,813 INFO org.apache.hadoop.ipc.Server: Stopping server on 9000
2013-01-19 00:34:55,814 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException: NameNode is not formatted.
at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:317)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:87)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:311)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:292)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:201)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:279)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:956)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965)
单击为分布式安装:
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/collector_3h/hadoop-0.20.2/hdfs/name</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/home/hadoop/collector_3h/hadoop-0.20.2/hdfs/data</value>
</property>
</configuration>
解决方案:
直接删除namenode的配置文件目录,在重新format
1,rm -rf ../hdfs/na*
2,./hadoop namenode -format
再次jps
hadoop@ubuntu:~/collector_3h/hadoop-0.20.2/bin$ jps
15742 JobTracker
15662 SecondaryNameNode
15499 DataNode
15895 TaskTracker
15335 NameNode
16127 Jps
hadoop@ubuntu:~/collector_3h/hadoop-0.20.2/bin$
2,$ bin/hadoop jar hadoop-0.20.2-examples.jar wordcount in out
java.io.FileNotFoundException: File e:/tmp/hadoop-SYSTEM/mapred/local/taskTracker/jobcache/job_xxxx/attempt_xxxxx/work/tmp does not exist
mapred-site.xml添加
<property>
<name>mapred.child.tmp</name>
<value>/tmp</value> <!-- 在mapred-default.xml中默认是./tmp -->
</property>
3,Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS: hdfs://localhost:9101/user/Administrator/hdfs_dir, expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:310)
at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:47)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:357)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:648)
at fs.Read.createdir(Read.java:43)
at fs.Read.main(Read.java:59)
cat hadoop-hadoop-namenode-ubuntu.log:
2013-01-19 00:34:55,812 ERROR org.apache.hadoop.hdfs.server.namenode.FSNamesystem: FSNamesystem initialization failed.
java.io.IOException: NameNode is not formatted.
at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:317)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:87)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:311)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:292)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:201)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:279)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:956)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965)
2013-01-19 00:34:55,813 INFO org.apache.hadoop.ipc.Server: Stopping server on 9000
2013-01-19 00:34:55,814 ERROR org.apache.hadoop.hdfs.server.namenode.NameNode: java.io.IOException: NameNode is not formatted.
at org.apache.hadoop.hdfs.server.namenode.FSImage.recoverTransitionRead(FSImage.java:317)
at org.apache.hadoop.hdfs.server.namenode.FSDirectory.loadFSImage(FSDirectory.java:87)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.initialize(FSNamesystem.java:311)
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.<init>(FSNamesystem.java:292)
at org.apache.hadoop.hdfs.server.namenode.NameNode.initialize(NameNode.java:201)
at org.apache.hadoop.hdfs.server.namenode.NameNode.<init>(NameNode.java:279)
at org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:956)
at org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:965)
单击为分布式安装:
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.permissions</name>
<value>false</value>
</property>
<property>
<name>dfs.name.dir</name>
<value>/home/hadoop/collector_3h/hadoop-0.20.2/hdfs/name</value>
</property>
<property>
<name>dfs.data.dir</name>
<value>/home/hadoop/collector_3h/hadoop-0.20.2/hdfs/data</value>
</property>
</configuration>
解决方案:
直接删除namenode的配置文件目录,在重新format
1,rm -rf ../hdfs/na*
2,./hadoop namenode -format
再次jps
hadoop@ubuntu:~/collector_3h/hadoop-0.20.2/bin$ jps
15742 JobTracker
15662 SecondaryNameNode
15499 DataNode
15895 TaskTracker
15335 NameNode
16127 Jps
hadoop@ubuntu:~/collector_3h/hadoop-0.20.2/bin$
2,$ bin/hadoop jar hadoop-0.20.2-examples.jar wordcount in out
java.io.FileNotFoundException: File e:/tmp/hadoop-SYSTEM/mapred/local/taskTracker/jobcache/job_xxxx/attempt_xxxxx/work/tmp does not exist
mapred-site.xml添加
<property>
<name>mapred.child.tmp</name>
<value>/tmp</value> <!-- 在mapred-default.xml中默认是./tmp -->
</property>
3,Exception in thread "main" java.lang.IllegalArgumentException: Wrong FS: hdfs://localhost:9101/user/Administrator/hdfs_dir, expected: file:///
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:310)
at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:47)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:357)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:245)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:648)
at fs.Read.createdir(Read.java:43)
at fs.Read.main(Read.java:59)
相关文章推荐
- Git 一些错误的解决方法 Pull is not possible because you have unmerged files.
- 大数据学习系列之八----- Hadoop、Spark、HBase、Hive搭建环境遇到的错误以及解决方法
- 关于Tomcat一些启动错误的解决方法
- 关于git的一些错误解决方法。
- 总结一些关于fatal error RC1004: unexpected end of file found错误的解决方法
- 安装CentOS碰到一错误的解决方法(图)
- Hadoop常见错误问题及解决方法(1)
- SqlServer try catch 捕获不到的一些错误及解决方法
- Git 一些错误的解决方法
- PCL库使用中出现的一些错误及解决方法
- 我遇到的hadoop错误和解决方法汇总
- MySql的一些错误解决方法
- 关于mini2440挂载nfs文件系统一些错误的解决方法
- hadoop配置lzo,外加自己遇到错误的解决方法
- Linux下安装Oracle出现的一些错误,以及解决方法
- win7中的eclipse连接虚拟机中Linux(Ubuntu)的Hadoop (附上配置时遇到的错误和解决的方法)
- Hadoop本地库与系统版本不一致引起的错误解决方法
- C程序访问hadoop出现的各种编译错误和运行时各种类DefFound异常的解决方法
- 大数据学习系列之八----- Hadoop、Spark、HBase、Hive搭建环境遇到的错误以及解决方法
- 关于Tomcat一些启动错误的解决方法