Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Changelog for Instructions to Reindex Indexes #6151

Merged
merged 12 commits into from
Oct 27, 2023

Conversation

AmitPhulera
Copy link
Contributor

Changelog for steps to reindex the ES indices.

Environments Affected

All

Announce New Release
  • After merging, I will follow these instructions
    to announce a new commcare-cloud release.

Comment on lines 21 to 28
- If you have sepearate data nodes, check disk usage on data nodes
```sh
cchq <env> run-shell-command es_data "df -h /opt/data" -b
```
- If you don't have separate data nodes, check disk usage on ES nodes.
```sh
cchq <env> run-shell-command es_data "df -h /opt/data" -b
```
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The commands in these two points are identical. If that was expected, would it be possible to combine the two points into one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope. It should have been

      cchq <env>  run-shell-command elasticsearch "df -h /opt/data" -b

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changelog/0075-reindex-all-indexes-for-es-upgrade.yml Outdated Show resolved Hide resolved

3. Verify reindex is completed by querying the logs and ensuring that doc count matches between primary and secondary indices. The commands in this step should be run from control machine.

1. In a new shell login to control machine.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. In a new shell login to control machine.
1. Login to the control machine in a new shell.

Also, formatting of this sub-list does not seem to be rendering correctly on github. It's rendering as a fixed-width code block, which might mean it's indented too far?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed the formatting in 65ed26f

changelog/0075-reindex-all-indexes-for-es-upgrade.yml Outdated Show resolved Hide resolved
changelog/0075-reindex-all-indexes-for-es-upgrade.yml Outdated Show resolved Hide resolved
@@ -0,0 +1,85 @@
title: reindex-all-indexes-for-es-upgrade
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I think the title field can be more "human readable" right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought this is what is used in urls but I realised now that we have key for that. Fixed in 75743be

changelog/0075-reindex-all-indexes-for-es-upgrade.yml Outdated Show resolved Hide resolved
changelog/0075-reindex-all-indexes-for-es-upgrade.yml Outdated Show resolved Hide resolved
changelog/0075-reindex-all-indexes-for-es-upgrade.yml Outdated Show resolved Hide resolved
changelog/0075-reindex-all-indexes-for-es-upgrade.yml Outdated Show resolved Hide resolved
changelog/0075-reindex-all-indexes-for-es-upgrade.yml Outdated Show resolved Hide resolved
changelog/0075-reindex-all-indexes-for-es-upgrade.yml Outdated Show resolved Hide resolved
[2023-10-25 08:59:37,648][INFO] [tasks] 29216 finished with response ReindexResponse[took=1.8s,updated=0,created=1111,batches=2,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,indexing_failures=[],search_failures=[]]
```

Ensure that `search_failures` and `indexing_failures` are empty lists.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a thought, but what should they do if either of these are not empty lists?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh! I missed it, thanks for catching that - added in a47409d

changelog/0075-reindex-all-indexes-for-es-upgrade.yml Outdated Show resolved Hide resolved

If reindex fails for any index, please refer to the docs [here](https://github.com/dimagi/commcare-hq/blob/56682492f20c60cdef0ccde6049b9945b3658493/corehq/apps/es/REINDEX_PROCESS.md#common-issues-resolutions-during-reindex) for troubleshooting steps.

4. If doc counts match and there are no errors present in the reindex logs, the reindex for the current index is complete. You can continue reindexing for the next index by repeating steps 5.1-5.3 with `${INDEX_CNAME}` set to 'cases', 'case_search' and so on for all indices.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: add newline

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@AmitPhulera AmitPhulera merged commit 5f65e4b into master Oct 27, 2023
@AmitPhulera AmitPhulera deleted the ap/es/reindex-changelog branch October 27, 2023 04:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants