Fatal error encountered during data read.
2017-12-20 15:58
706 查看
语言:C# (主要技术:SqlBulkCopy)
场景:MySQL => SQL Server
需求:将MySQL中的一个 1186264 行( 纯数据大小 2.8 GB )的表所有记录传输到 SQL Server 的表中。
发现异常后处理过程:
1. 先将sql语句前面加上:"set net_write_timeout=99999; set net_read_timeout=99999;" , 异常依旧;
2. 发现大约执行 10 分钟后产生了异常,于是在 MySQL 中执行: " set global wait_timeout=99999; " , 异常依旧;
3. 设置 BulkSize = 1000 ,再执行程序发现SQL Server表中查到记录数在增长,但大约传输 38 万行数据后不再有数据增加, 近一个小时还没动静,手动停止了;
4. MySQL中增加一个新表 dc_init_temp
5. 将表中主键插入到新表中;
6. 采用分页方式循环将记录传输到SQL Server:
7. 到此终于解决, 同步完成 634 秒(当然,因为是测试环境的虚拟机,IO差一点,正式服务器会更快)。
场景:MySQL => SQL Server
需求:将MySQL中的一个 1186264 行( 纯数据大小 2.8 GB )的表所有记录传输到 SQL Server 的表中。
发现异常后处理过程:
1. 先将sql语句前面加上:"set net_write_timeout=99999; set net_read_timeout=99999;" , 异常依旧;
2. 发现大约执行 10 分钟后产生了异常,于是在 MySQL 中执行: " set global wait_timeout=99999; " , 异常依旧;
3. 设置 BulkSize = 1000 ,再执行程序发现SQL Server表中查到记录数在增长,但大约传输 38 万行数据后不再有数据增加, 近一个小时还没动静,手动停止了;
4. MySQL中增加一个新表 dc_init_temp
drop table if exists dc_init_temp; create table dc_init_temp( pkId varchar(100) not null, autoId INT(10) AUTO_INCREMENT, PRIMARY KEY (`pkId`), KEY `ix_dc_init_temp_autoId` (`autoId`) );
5. 将表中主键插入到新表中;
6. 采用分页方式循环将记录传输到SQL Server:
set net_write_timeout=99999; set net_read_timeout=99999; SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED; select * from dbName.tableName as a WHERE a.id in ( SELECT b.`pkId` FROM `dc_init_temp` as b where b.autoId>? and b.autoId<=? #(pageIndex-1)*pageSize , pageIndex*pageSize ); COMMIT;
7. 到此终于解决, 同步完成 634 秒(当然,因为是测试环境的虚拟机,IO差一点,正式服务器会更快)。
相关文章推荐
- MYSQL报Fatal error encountered during command execution.错误的解决方法
- Weblogic安装出错Fatal error encountered during file installation
- Fatal error encountered during command execution
- mysql报Fatal error encountered during command execution的解决办法
- mysql报Fatal error encountered during command execution的解决办法
- Fatal error encountered during command execution
- Fatal error encountered during command execution
- My SQL和LINQ 实现ROW_NUMBER() OVER以及Fatal error encountered during command execution
- mysql报Fatal error encountered during command execution的解决办法
- mysql报Fatal error encountered during command execution的解决办法
- 异常诡异的mysql连接异常:Fatal error encountered during command execution.
- 可遇不可求的Question之Fatal error encountered during command execution.
- [生产环境数据恢复]innobackupex: fatal error: OR no 'datadir' option in group 'mysqld' in MySQL options
- uploadify 报错jquery.uploadify.min.js:16 Uncaught TypeError: Cannot read property 'queueData' of undef
- kafka启动时出现FATAL Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer) java.io.IOException: Permission denied错误解决办法(图文详解)
- C++编程故障处理——error: assignment of read-only data-member ‘std::pair
- The server has encountered an error while loading an application during the processing of your reque
- No more data to read from socket error
- ERROR: Error in Log_event::read_log_event(): 'read error', data_len: 438, event_type: 2
- ERROR: Error in Log_event::read_log_event(): 'read error', data_len: 438, event_type: 2