Add automatic migration path from local to replicated metadata server #2708
Conversation
| if raft_options.migrate_from_local_metadata_server { | ||
| info!("Trying to migrate from local metadata to replicated metadata server"); | ||
|
|
||
| let my_member_id = self | ||
| .initialize_storage_from_local_metadata_server() | ||
| .await | ||
| .map_err(|err| Error::ProvisionFromLocal(err.into()))?; | ||
|
|
||
| info!(member_id = %my_member_id, "Successfully migrated all data from local metadata to replicated metadata server"); | ||
| let member = self.become_member(my_member_id)?; | ||
| Provisioned::Member(member) |
There was a problem hiding this comment.
I wonder if we should make this migration enabled by default (or not being an option). Which one is more risky?
There was a problem hiding this comment.
I thought that making the migration explicit and requiring people to opt-in to it, it gives a bit more control over what is happening. I guess the situation it protects against is that you have a local-metadata-store directory in your node directory from which you don't want to migrate. Admittedly, it makes changing metadata-server types less convenient since it requires users to read the documentation.
There was a problem hiding this comment.
What if I started the server and didn't actually migrate? what happens to the state of current data in loglets/partitions? I'm concerned about this leading to irreparable corrupted state.
There was a problem hiding this comment.
Very good point. If the nodes start with a fresh metadata, then they will start again from lsn 0 to write to the log. I will change it to auto migrate.
tillrohrmann
force-pushed
the
local-to-replicated-migration
branch
3 times, most recently
from
46c3319 to
8ef9255
Compare
|
I've changed the logic to work w/o having to set an explicit option. If the local metadata data directory exists, then the system will try to migrate from it. |
tillrohrmann
force-pushed
the
local-to-replicated-migration
branch
from
8ef9255 to
5c9e781
Compare
Instead of creating a log entry with a WriteRequest, this commit changes how the RaftMetadataServer is initialized by using a KvMemoryStorage snapshot instead.
This commit adds an automatic migraton path from a local to a replicated metadata server. To make use of it, users need to set metadata-server.type = replicated metadata-server.migrate_from_local_metadata_server = true If the replicated metadata server hasn't provisioned before, then it will read the key-value pairs from the local metadata store and initialize its own storage with it. It will also run the migration logic for older NodesConfiguration versions (those that might still have a node id of 0 configured).
…g to replicated metadata server This is a safety mechanism that should prevent accidental misconfiguration.
tillrohrmann
force-pushed
the
local-to-replicated-migration
branch
from
5c9e781 to
9a1d297
Compare
This commit adds an automatic migration path from a local to a replicated metadata
server. To make use of it, users need to set
metadata-server.type = replicated
metadata-server.migrate_from_local_metadata_server = true
If the replicated metadata server hasn't provisioned before, then it will read the
key-value pairs from the local metadata store and initialize its own storage with it.
It will also run the migration logic for older NodesConfiguration versions (those
that might still have a node id of 0 configured).