[BUG] indexer broken after adding username and password to the keystore for notifications

[Orgoth]

Describe the bug
Tried to set up notifications as recommended by the dashboard!
But after adding username and passwort to the opensearch keystore, the whole indexer is broken.

To Reproduce
Steps to reproduce the behavior:

1. Go to /app/notifications-dashboards#/create-smtp-sender
2. create a new sender uniquename "wazu_01", smtp domain, email, port 465
3. do not follow the learn more link, since it is outdated "https://opensearch.org/docs/monitoring-plugins/alerting/monitors/#authenticate-sender-account" and redirects to "https://opensearch.org/docs/latest/observing-your-data/alerting/monitors/" which does not help at all.
4. therefore i followed this link: https://opensearch.org/docs/latest/observing-your-data/notifications/index/#authenticate-sender-account
5. added user and password to the keystore

/usr/share/wazuh-indexer/bin/opensearch-keystore add opensearch.notifications.core.email.wazu_01.username
/usr/share/wazuh-indexer/bin/opensearch-keystore add opensearch.notifications.core.email.wazu_01.password

6. restarted indexer

Expected behavior
Indexer should have started without any problems, since I have only added data for wazu_01 and nothing else!

Errors

Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]: Exception in thread "main" java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:224)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at java.base/java.nio.channels.FileChannel.open(FileChannel.java:309)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at java.base/java.nio.channels.FileChannel.open(FileChannel.java:369)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:156)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at org.apache.lucene.backward_codecs.store.EndiannessReverserUtil.openChecksumInput(EndiannessReverserUtil.java:48)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:248)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:232)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at org.opensearch.common.settings.HasPasswordKeyStoreCommand.execute(HasPasswordKeyStoreCommand.java:61)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at org.opensearch.cli.MultiCommand.execute(MultiCommand.java:104)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at org.opensearch.cli.Command.main(Command.java:101)
Nov 15 07:22:31 wazuh.somedomain.de systemd-entrypoint[164660]:         at org.opensearch.common.settings.KeyStoreCli.main(KeyStoreCli.java:56)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]: WARNING: A terminally deprecated method in java.lang.System has been called
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]: WARNING: System::setSecurityManager has been called by org.opensearch.bootstrap.OpenSearch (file:/usr/share/wazuh-indexer/lib/opensearch-2.13.0.jar)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]: WARNING: Please consider reporting this to the maintainers of org.opensearch.bootstrap.OpenSearch
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]: WARNING: System::setSecurityManager will be removed in a future release
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]: Exception in thread "main" org.opensearch.bootstrap.BootstrapException: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]: Likely root cause: java.nio.file.AccessDeniedException: /etc/wazuh-indexer/opensearch.keystore
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at java.base/sun.nio.fs.UnixException.translateToIOException(UnixException.java:90)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at java.base/sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at java.base/sun.nio.fs.UnixFileSystemProvider.newFileChannel(UnixFileSystemProvider.java:224)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at java.base/java.nio.channels.FileChannel.open(FileChannel.java:309)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at java.base/java.nio.channels.FileChannel.open(FileChannel.java:369)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:78)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:156)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at org.apache.lucene.backward_codecs.store.EndiannessReverserUtil.openChecksumInput(EndiannessReverserUtil.java:48)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:248)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at org.opensearch.common.settings.KeyStoreWrapper.load(KeyStoreWrapper.java:232)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at org.opensearch.bootstrap.Bootstrap.loadSecureSettings(Bootstrap.java:257)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at org.opensearch.bootstrap.Bootstrap.init(Bootstrap.java:368)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at org.opensearch.bootstrap.OpenSearch.init(OpenSearch.java:181)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at org.opensearch.bootstrap.OpenSearch.execute(OpenSearch.java:172)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at org.opensearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:104)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at org.opensearch.cli.Command.mainWithoutErrorHandling(Command.java:138)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at org.opensearch.cli.Command.main(Command.java:101)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:138)
Nov 15 07:22:32 wazuh.somedomain.de systemd-entrypoint[164604]:         at org.opensearch.bootstrap.OpenSearch.main(OpenSearch.java:104)

Host/Environment (please complete the following information):

• OS: Ubuntu 24.04 LTS
• Wazuh 4.9.2

Additional context

I have a postfix smarthost running which works completely fine for all CLI related notifications.
For example: echo "Test mail from postfix" | mail -s "Test Postfix" -r "<CONFIGURED_EMAIL>" <RECEIVER_EMAIL>
Works just fine.
But the Dashboard Notifications are using a completely different notifications service then the wazu manager.

[AlexRuiz7]

Hello @Orgoth

You can revert the situation by removing the keystore. Be sure to make a backup before doing so:

sudo rm /etc/wazuh-indexer/opensearch-keystore

I'll try to reproduce the issue.