Clustering-service cannot be deployed on read-only container filesystem

[rbogendoerfer]

Describe the bug
Some clients are enforcing the container filesystem to be read-only. In these situations the clustering-service fails to startup with the following message:

2024/11/21 08:46:30 failed to process template file /opt/hazelcast/templates/hazelcast.xml.tmpl, error: invalid argument

To Reproduce
This behavior can be caused by enforcing read-only root filesystem. To do that just add the containerSecurityContext to the clustering-service deployment:

  containerSecurityContext:
    allowPrivilegeEscalation: false
    readOnlyRootFilesystem: true

Please note that those kind of settings are enforced by the security departments of many clients!

Expected behavior
Either the clustering-service should support read-only root filesystem out-of-the-box or it should allow to provide custom volumes and initContainers (like the pega deployment) to workaround the issue.

Chart version
3.25.0 without any changes

Server (if applicable, please complete the following information):

• OS: CentOS Stream 9
• Environment: Open Source Kubernetes 1.31.2
• Database: PostgreSQL 14

Additional context
The clustering service already uses an emptyDir volume for the log directory.
However during startup the hazelcast.xml.tmpl gets rewritten by detemplatize to the customconfig directory, which is on the root filesystem. When the root filesystem is read-only, this will fail.
A clean solution would be to make the customcofig directory also an emptyDir volume and use the entrypoint script to copy or rewrite (detemplatize) all configuration files into it.
Allowing custom volumes and initContainers like in the pega deployment would be another solution.

[misterdorito]

@vnihal72 can you take a look at this issue? Thanks.