sqoop导出hive数据到mysql错误ava.util.NoSuchElementException解决办法
2014-05-08 12:39
567 查看
sqoop错误日志:
14/05/08 12:25:40 INFO mapred.JobClient: Task Id : attempt_201405071437_0013_m_000000_0, Status : FAILED
java.io.IOException: Can't export data, please check task tracker logs
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(ArrayList.java:794)
at wordcount.__loadFromFields(wordcount.java:198)
at wordcount.parse(wordcount.java:147)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83)
... 10 more
14/05/08 12:25:40 INFO mapred.JobClient: Task Id : attempt_201405071437_0013_m_000001_0, Status : FAILED
java.io.IOException: Can't export data, please check task tracker logs
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(ArrayList.java:794)
at wordcount.__loadFromFields(wordcount.java:198)
at wordcount.parse(wordcount.java:147)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83)
... 10 more
sqoop导出语句:
sqoop export --connect jdbc:mysql://icity0:3306/sqoop --username root --password root --table wordcount --input-fields-terminated-by '\001' --export-dir '/user/hive/warehouse/wordcount'
这个错误的原因是指定Hive中表字段之间使用的分隔符错误,供Sqoop读取解析不正确,由于此处hive中的数据是由hdfs中导入的,其分隔符是'\t',故修改从下列的导出语句即可:
sqoop export --connect jdbc:mysql://icity0:3306/sqoop --username root --password root --table wordcount --input-fields-terminated-by '\t' --export-dir '/user/hive/warehouse/wordcount'
注:如果是由hive执行mapreduce操作汇总的结果,默认的分隔符是 '\001'.
14/05/08 12:25:40 INFO mapred.JobClient: Task Id : attempt_201405071437_0013_m_000000_0, Status : FAILED
java.io.IOException: Can't export data, please check task tracker logs
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(ArrayList.java:794)
at wordcount.__loadFromFields(wordcount.java:198)
at wordcount.parse(wordcount.java:147)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83)
... 10 more
14/05/08 12:25:40 INFO mapred.JobClient: Task Id : attempt_201405071437_0013_m_000001_0, Status : FAILED
java.io.IOException: Can't export data, please check task tracker logs
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.util.NoSuchElementException
at java.util.ArrayList$Itr.next(ArrayList.java:794)
at wordcount.__loadFromFields(wordcount.java:198)
at wordcount.parse(wordcount.java:147)
at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83)
... 10 more
sqoop导出语句:
sqoop export --connect jdbc:mysql://icity0:3306/sqoop --username root --password root --table wordcount --input-fields-terminated-by '\001' --export-dir '/user/hive/warehouse/wordcount'
这个错误的原因是指定Hive中表字段之间使用的分隔符错误,供Sqoop读取解析不正确,由于此处hive中的数据是由hdfs中导入的,其分隔符是'\t',故修改从下列的导出语句即可:
sqoop export --connect jdbc:mysql://icity0:3306/sqoop --username root --password root --table wordcount --input-fields-terminated-by '\t' --export-dir '/user/hive/warehouse/wordcount'
注:如果是由hive执行mapreduce操作汇总的结果,默认的分隔符是 '\001'.
相关文章推荐
- 使用Sqoop,最终导入到hive中的数据和原数据库中数据不一致解决办法
- 在使用中文名导出数据到excel文件时文件名出现乱码的解决办法
- sqoop从关系库导出数据到hive
- sqoop导出hive表数据到mysql
- 在使用 ADO.NET 导出 Excel 文件时,设置 IMEX=1 仍不能导出所有数据的解决办法
- [Sqoop]将Hive数据表导出到Mysql
- 利用sqoop将hive数据导入导出数据到mysql
- Oozie配合Hive、Sqoop定时导出Hive表数据至RDB
- 使用sqoop将hive中的表数据导入到mysql数据库表中,错误解决
- PowerDesiger 小写 导出数据表 sql 出现 引号 解决办法
- 用PL/SQL Developer导出表数据的时候,窗口一闪而过解决办法
- mysql导出数据时出现--secure-file-priv 问题的解决办法
- EXP-00079: 表 "***" 中的数据是被保护的。常规路径只能导出部分表的解决办法
- java导出excel大量数据出现错误时解决办法
- Hadoop记录——hive中的decimal字段,shell的数据类型以及sqoop导出空string报错
- 6.(Mysql数据管理相关)连接MYSQL,修改密码,增加新用户,数据库相关命令,表操作相关命令,数据相关命令,数据库sql导入和导出,备份数据库,查看不到mysql数据库的解决办法
- 从Hive导出数据到Oracle数据库--Sqoop
- 用PL/SQL Developer导出表数据的时候,窗口一闪而过解决办法
- Hive数据导入、sqoop数据导入导出
- 使用sqoop实现关系型数据库、HDFS、Hive之间数据的导入导出