OplogThread: Last entry no longer in oplog cannot recover! - Resyncing the Connector
2017-09-29 17:32
1071 查看
mongo-connector 和 es同步数据时出现如下错误
mongo_connector.oplog_manager:202 - OplogThread: Last entry no longer in oplog cannot recover! Collection(Database(MongoClient(host=
可能需要
https://github.com/mongodb-labs/mongo-connector/wiki/Resyncing-the-Connector
This page describes when and how to re-sync mongo-connector. You'll need to do this if you see the error message Last entry no longer in oplog cannot recover!. The most common reason to need to re-sync mongo-connector is that it couldn't
replicate operations from the oplog fast enough. This can happen when there is a lot of write activity happening in MongoDB, such as when using
Because the oplog is a capped collection, older records are overwritten when the collection is full.
Mongo-connector can be more tolerant to short bursts of high write activity by increasing the oplog size in MongoDB.
The greater oplog time allows mongo-connector to "catch up" when there is less write activity.
The only way to ensure that the data in your external system is consistent with what is in MongoDB is to delete and re-index all documents in the target. After all data is removed, you may delete the oplog progress file (usually called "oplog.timestamp") and
re-start mongo-connector. Mongo-connector will then perform a collection dump, re-indexing all your data. Be careful and double-check that you are deleting only and exactly what you mean to delete.
The simplest and fastest way to remove data from MongoDB is to drop the database:
Or only drop a collection:
You can remove all data by sending a GET request to a URL:
You can remove all data quickly and efficiently by deleting the index and
re-creating it:
After this, you should refresh the index to make these changes visible:
There aren't any other methods to restore a consistent state with the source MongoDB replica set or cluster. However, you can get mongo-connector simply running again by deleting the oplog progress file and restarting mongo-connector. This causes mongo-connector
to perform a collection dump, re-saving the latest versions of all documents, then start tailing the oplog. This does not bring your target to a consistent state but may be suitable for pure insert/update use cases. If
any delete operations were clobbered by the oplog collection rollover, mongo-connector cannot catch them without a proper re-sync (described above)
mongo_connector.oplog_manager:202 - OplogThread: Last entry no longer in oplog cannot recover! Collection(Database(MongoClient(host=
可能需要
Resyncing the Connector
https://github.com/mongodb-labs/mongo-connector/wiki/Resyncing-the-ConnectorThis page describes when and how to re-sync mongo-connector. You'll need to do this if you see the error message Last entry no longer in oplog cannot recover!. The most common reason to need to re-sync mongo-connector is that it couldn't
replicate operations from the oplog fast enough. This can happen when there is a lot of write activity happening in MongoDB, such as when using
mongoimport.
Because the oplog is a capped collection, older records are overwritten when the collection is full.
Avoiding
Oplog Rollover
Mongo-connector can be more tolerant to short bursts of high write activity by increasing the oplog size in MongoDB.The greater oplog time allows mongo-connector to "catch up" when there is less write activity.
How
to Perform a Re-Sync
The only way to ensure that the data in your external system is consistent with what is in MongoDB is to delete and re-index all documents in the target. After all data is removed, you may delete the oplog progress file (usually called "oplog.timestamp") andre-start mongo-connector. Mongo-connector will then perform a collection dump, re-indexing all your data. Be careful and double-check that you are deleting only and exactly what you mean to delete.
MongoDB
The simplest and fastest way to remove data from MongoDB is to drop the database:mongo > db.getSisterDB("<database name>").dropDatabase() { "dropped" : "<database name>", "ok" : 1 }
Or only drop a collection:
> db.getSisterDB("<database name>").<collection name>.drop() true
Solr
You can remove all data by sending a GET request to a URL:http://<hostname>:<port>/solr/<core name>/update?commit=true&stream.body=<delete><query>*:*</query></delete>
Elasticsearch
You can remove all data quickly and efficiently by deleting the index andre-creating it:
curl -XDELETE http://<hostname>:<port>/<index name> curl -XPUT http://<hostname>:<port>/<index name>
After this, you should refresh the index to make these changes visible:
curl -XPOST http://<hostname>:<port>/<index name>/_refresh
Alternatives
There aren't any other methods to restore a consistent state with the source MongoDB replica set or cluster. However, you can get mongo-connector simply running again by deleting the oplog progress file and restarting mongo-connector. This causes mongo-connectorto perform a collection dump, re-saving the latest versions of all documents, then start tailing the oplog. This does not bring your target to a consistent state but may be suitable for pure insert/update use cases. If
any delete operations were clobbered by the oplog collection rollover, mongo-connector cannot catch them without a proper re-sync (described above)
相关文章推荐
- SharePoint Designer Check In and Check Out Error – Cannot perform this operation.The file is no longer check out or has been del
- ArcMap 连接SDE 出错“Failed to connect to the specified server. Entry for SDE instance no found in services file.”
- 20110323_BMXAA7901E - You cannot log in at this time. Contact the system administrator
- 【转】解决:ORA-19602: cannot backup or copy active file in NOARCHIVELOG mode
- 用web 打开phpMyAdmin时出现错误#2002 Cannot log in to the MySQL server
- in thread "main" ImportError: Cannot import site module and its dependencies: No module named site
- 【转】解决:ORA-19602: cannot backup or copy active file in NOARCHIVELOG mode
- ORA-19602: cannot backup or copy active file in NOARCHIVELOG mode
- RMAN-06817: Pluggable Database CHARLESPDB cannot be backed up in NOARCHIVELOG mode.
- The project file ' ' has been renamed or is no longer in the solution. 解决方案
- The transaction log for database 'NJ_Address' is full. To find out why space in the log cannot be
- php出现[#2002 Cannot log in to the MySQL server]解决方法
- Error: Flag android.useDeprecatedNdk is no longer supported and will be removed in the next version
- React Native demo运行报错:Cannot find entry file index.android.js in any of the roots
- The project file '' has been renamed or is no longer in the solution 解决方案
- the project file '' has been renamed or is no longer in the solution 解决办法
- VS2010 The project file ' ' has been renamed or is no longer in the solution
- Given constant integers x and t, write a function that takes no argument and returns true if the function has been called x number of times in last t secs.
- 【原】Cannot find entry file index.ios.js [index.android.js] in any of the roots
- phpmyadmin 错误 #2002 Cannot log in to the MySQL server