您的位置:首页 > 数据库 > SQL

sqoop导出hive数据到mysql错误ava.util.NoSuchElementException解决办法

2014-05-08 12:39 567 查看
sqoop错误日志:

14/05/08 12:25:40 INFO mapred.JobClient: Task Id : attempt_201405071437_0013_m_000000_0, Status : FAILED

java.io.IOException: Can't export data, please check task tracker logs

at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)

at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)

at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)

at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)

at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)

at org.apache.hadoop.mapred.Child.main(Child.java:249)

Caused by: java.util.NoSuchElementException

at java.util.ArrayList$Itr.next(ArrayList.java:794)

at wordcount.__loadFromFields(wordcount.java:198)

at wordcount.parse(wordcount.java:147)

at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83)

... 10 more

14/05/08 12:25:40 INFO mapred.JobClient: Task Id : attempt_201405071437_0013_m_000001_0, Status : FAILED

java.io.IOException: Can't export data, please check task tracker logs

at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:112)

at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:39)

at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)

at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:64)

at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:415)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)

at org.apache.hadoop.mapred.Child.main(Child.java:249)

Caused by: java.util.NoSuchElementException

at java.util.ArrayList$Itr.next(ArrayList.java:794)

at wordcount.__loadFromFields(wordcount.java:198)

at wordcount.parse(wordcount.java:147)

at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:83)

... 10 more

sqoop导出语句:

sqoop export --connect jdbc:mysql://icity0:3306/sqoop --username root --password root --table wordcount --input-fields-terminated-by '\001' --export-dir '/user/hive/warehouse/wordcount'

这个错误的原因是指定Hive中表字段之间使用的分隔符错误,供Sqoop读取解析不正确,由于此处hive中的数据是由hdfs中导入的,其分隔符是'\t',故修改从下列的导出语句即可:

sqoop export --connect jdbc:mysql://icity0:3306/sqoop --username root --password root --table wordcount --input-fields-terminated-by '\t' --export-dir '/user/hive/warehouse/wordcount'

注:如果是由hive执行mapreduce操作汇总的结果,默认的分隔符是 '\001'.
内容来自用户分享和网络整理,不保证内容的准确性,如有侵权内容,可联系管理员处理 点击这里给我发消息
标签: 
相关文章推荐