Sqoop 数据导入问题
2016-04-22 12:54
776 查看
今天在使用sqoop将数据导入到hive中时一直遇到这个问题
经过google得知原来是mysql的jdbc驱动包不匹配导致的。
http://stackoverflow.com/questions/29162447/sqoop-import-issue-with-mysql
Try with mysql-connector-java-5.1.31.jar, it is compatable with sqoop
1.4.5.
mysql-connector-java-5.1.17.jar driver does not work with sqoop 1.4.5.
refer :
https://issues.apache.org/jira/browse/SQOOP-1400
sqoop import --connect jdbc:mysql://172.22.35.145:3306/test --username hive --P --table student --fields-terminated-by '\t' -m 1 --target-dir=/hive/warehouse/
16/04/22 12:45:47 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6 Enter password: 16/04/22 12:45:49 INFO manager.MySQLManager: Preparing to use a MySQL streaming resultset. 16/04/22 12:45:49 INFO tool.CodeGenTool: Beginning code generation 16/04/22 12:45:50 INFO manager.SqlManager: Executing SQL statement: SELECT t.* FROM `student` AS t LIMIT 1 16/04/22 12:45:50 ERROR manager.SqlManager: Error reading from database: java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@61220ed4 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries. java.sql.SQLException: Streaming result set com.mysql.jdbc.RowDataDynamic@61220ed4 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries. at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:934) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:931) at com.mysql.jdbc.MysqlIO.checkForOutstandingStreamingData(MysqlIO.java:2735) at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1899) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2619) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2569) at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1521) at com.mysql.jdbc.ConnectionImpl.getMaxBytesPerChar(ConnectionImpl.java:3003) at com.mysql.jdbc.Field.getMaxBytesPerCharacter(Field.java:602) at com.mysql.jdbc.ResultSetMetaData.getPrecision(ResultSetMetaData.java:445) at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:286) at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:241) at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:227) at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:295) at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833) at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645) at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605) at org.apache.sqoop.Sqoop.run(Sqoop.java:143) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) at org.apache.sqoop.Sqoop.main(Sqoop.java:236) 16/04/22 12:45:50 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: No columns to generate for ClassWriter at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1651) at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107) at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478) at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605) at org.apache.sqoop.Sqoop.run(Sqoop.java:143) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218) at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227) at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
经过google得知原来是mysql的jdbc驱动包不匹配导致的。
http://stackoverflow.com/questions/29162447/sqoop-import-issue-with-mysql
Try with mysql-connector-java-5.1.31.jar, it is compatable with sqoop
1.4.5.
mysql-connector-java-5.1.17.jar driver does not work with sqoop 1.4.5.
refer :
https://issues.apache.org/jira/browse/SQOOP-1400
相关文章推荐
- 发布网站配置IIS(把网上找到的解决方法综合了一下)
- OpenCV之core 模块. 核心功能(2)基本绘图 随机数发生器&绘制文字 离散傅立叶变换 输入输出XML和YAML文件 与 OpenCV 1 同时使用
- [OpenGL红宝书]第一章 OpenGL概述
- OpenCV之core 模块. 核心功能(1)Mat - 基本图像容器 OpenCV如何扫描图像、利用查找表和计时 矩阵的掩码操作 使用OpenCV对两幅图像求和(求混合(blending))
- Docker - Section II
- linux解压覆盖命令
- MMORPG服务器架构
- shell脚本中一些特殊符号
- 一个不错的shell 脚本教程 入门级
- Centos 6.5 安装icingaweb1 中文版部署
- tomcat-禁止某些IP访问的方法
- VMware和CentOS7安装和配置
- tomcat server.xml docbase workdir
- tomcat中引用js静态文件找不到问题
- shell 小技巧之修改后缀及grep
- 微服务架构与可持续交付实践
- linux下查看java 占用cpu使用情况
- IT运维流程 — ITIL
- opencv:求视频的前n帧的融合图像
- linux软件安装与卸载