Connector re-indexing everything after crash.

[syz3r]

Hi,

I'm running mongo:latest, elastic:latest and mongo-connector in a 64g/12c cloud instance. Sometimes connector crashes which is fine. But it re-indexes everything after restart which is quiet annoying. Is there is any option to disable that?

[syz3r]

Crash logs

Exception ignored in: <bound method Cursor.__del__ of <pymongo.cursor.Cursor object at 0x7f216bb36f28>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/pymongo/cursor.py", line 295, in __del__
    self.__die()
  File "/usr/local/lib/python3.4/site-packages/pymongo/cursor.py", line 360, in __die
    connection.close_cursor(self.__id)
  File "/usr/local/lib/python3.4/site-packages/pymongo/mongo_client.py", line 1524, in close_cursor
    self.__cursor_manager.close(cursor_id)
  File "/usr/local/lib/python3.4/site-packages/pymongo/cursor_manager.py", line 54, in close
    self.__connection().kill_cursors([cursor_id])
AttributeError: 'NoneType' object has no attribute 'kill_cursors'
Mongod node is now primary
Logging to mongo-connector.log.
Exception ignored in: <bound method Cursor.__del__ of <pymongo.cursor.Cursor object at 0x7ff0052a16d8>>
Traceback (most recent call last):
  File "/usr/local/lib/python3.4/site-packages/pymongo/cursor.py", line 295, in __del__
    self.__die()
  File "/usr/local/lib/python3.4/site-packages/pymongo/cursor.py", line 360, in __die
    connection.close_cursor(self.__id)
  File "/usr/local/lib/python3.4/site-packages/pymongo/mongo_client.py", line 1524, in close_cursor
    self.__cursor_manager.close(cursor_id)
  File "/usr/local/lib/python3.4/site-packages/pymongo/cursor_manager.py", line 54, in close
    self.__connection().kill_cursors([cursor_id])
AttributeError: 'NoneType' object has no attribute 'kill_cursors'

[yeasy]

Not sure, that will depend on mongo-connector's implementation.
Here is just a docker wrapper.

[mgebundy]

I believe it has to do with the oplog.timestamp
If that file gets reset when your connector crashes, that will re-sync the database. Maybe putting it in a shared volume will take care of that.

use the -v argument with docker run something like -v cdata:/data where cdata is the volume you created.

[yeasy]

Good advide, can u confirm it works?
If so, a pr is welcome :)