Add Migration Assistant Type Mapping documentation #9164
Conversation
Signed-off-by: Andre Kurait <[email protected]>
AndreKurait
requested review from
kolchfa-aws,
Naarcha-AWS,
AMoo-Miki,
natebower,
dlvenable and
epugh
as code owners
|
Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Editorial review -> Merged. Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer. When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). The doc reviewer will arrange for an editorial review. |
| 8. When running metadata migration, run with the additional parameter `console metadata migrate --transformer-config-file /shared-logs-output/transformation.json`.[^1] | ||
| 9. If the transformation configuration is updated, backfill/replayer will need to be stopped and restarted to apply the changes. | ||
|
|
||
| [^1]: The `/shared-logs-output` mount will soon be relocated with a new mountpoint on the container for the EFS volume. |
There was a problem hiding this comment.
Can we remove, what action would a reader of the document take with this note?
There was a problem hiding this comment.
Thanks for writing this up.
Lets consider reframing this documentation for a user they doesn't know if they use this deprecated feature, this kind of jumps into 'what to do' without enough context for a user to know if they are impacted and why they should make different choices.
| ### Using the TypeMappingsSanitizationTransformer | ||
|
|
||
| 1. Navigate to the bootstrap box and open the `cdk.context.json` with vim. | ||
| 2. Add/Update the key `reindexFromSnapshotExtraArgs` to include `--doc-transformer-config-file /shared-logs-output/transformation.json`. [^1] |
There was a problem hiding this comment.
Can we supply an empty file at this location and adjust our defaults to pull from this location - that would shorten what a user need to do? We are adding a lot of steps that look very similar and are easy to do incorrectly.
| "post": "new_posts" | ||
| } | ||
| }, | ||
| "sourceProperties": { |
There was a problem hiding this comment.
Can we remove the version here? Maybe I am missing how it is valuable for a user to pick a different version
| "user": "combined_activity", | ||
| "post": "combined_activity" |
There was a problem hiding this comment.
Can we simplify this name to activity?
| [ | ||
| { | ||
| "TypeMappingsSanitizationTransformerProvider": { | ||
| "staticMappings": { |
There was a problem hiding this comment.
Can we drop this node, so its just a list of indices and the types to be mapped?
| "TypeMappingsSanitizationTransformerProvider": { | ||
| "staticMappings": { | ||
| "activity": { | ||
| "user": "users_only", |
There was a problem hiding this comment.
So any types that are included will be dropped? That seems like an easy thing to get wrong or mistype, do we have a way to preflight this and warn users what they are doing if its by accident or is there a way we can make this more intention?
{
"{index-name}": {
"mappedTypes": {
"users": "users_only"
},
"droppedTypes": [
"post"
]
}
}| [ | ||
| "(.*)", | ||
| ".*", | ||
| "$1" | ||
| ] |
There was a problem hiding this comment.
Please add names property names to signal what these mean
| - Within the replayer, create index split is not supported. | ||
| - Note: This is only impactful on ES 5.x since multi type index creation is not supported on ES 6.x. |
There was a problem hiding this comment.
Lets frame this like the other operations so its easier to see.
If it is a known gap, we should proactively create a github issue that folks could comment / vote on and we'd leave a link here in the docs.
| - PUT/POST /{index}/{type}/{id} | ||
| - PUT/POST /{index}/{type}/ | ||
| - GET /{index}/{type}/{id} | ||
| - PUT/POST /_bulk | ||
| - PUT/POST /{index} |
There was a problem hiding this comment.
Lets turn this into a table with a leading column for the operation name such as Index, etc Bulk Index/Update/Delete.
| - Within the replayer, create index split is not supported. | ||
| - Note: This is only impactful on ES 5.x since multi type index creation is not supported on ES 6.x. | ||
|
|
||
| ### Important Notes |
There was a problem hiding this comment.
these important notes are at the bottom, lets see if we can better integrate them into the workflow.
Description
Add Migration Assistant Type Mapping documentation
Issues Resolved
MIGRATIONS-2385
Version
all (Migration Assistant 2.1.5+)
Frontend features
n/a
Checklist
For more information on following Developer Certificate of Origin and signing off your commits, please check here.