Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change HDFS settings to easily allow Accumulo init #246

Open
GCHQDeveloper314 opened this issue Aug 11, 2022 · 0 comments
Open

Change HDFS settings to easily allow Accumulo init #246

GCHQDeveloper314 opened this issue Aug 11, 2022 · 0 comments
Labels
Docker Issue related to the Docker side of the project good first issue Small, lower complexity and doesn't require pre-existing Gaffer knowledge tech debt Relates to Technical Debt
Milestone

Comments

@GCHQDeveloper314
Copy link
Member

In all docker-compose.yaml files that involve Accumulo, there's this comment on each Accumulo service:

# There doesn't seem to be an easy way (with docker-compose) to init our
# HDFS instance with the right permissions so that Accumulo can create the
# file structure it needs. Using the following workaround to allow
# accumulo to "auth" with HDFS as the super user so that it can:
- HADOOP_USER_NAME=hadoop

For Kubernetes the following is done to grant "the right permissions":

- hadoop fs -mkdir -p /accumulo
- hadoop fs -chown accumulo /accumulo
- hadoop fs -chmod 700 /accumulo
- hadoop fs -ls /

The claim that this cannot be done in docker-compose doesn't seem to be correct. It should be possible to run the same commands in the HDFS container when using docker-compose.

However, seeing as secure mode is not in use and the only client is Accumulo, there's very little point in using HDFS permissions. Disabling them (by setting dfs.permissions.enabled to false) removes the need for this workaround and this tedious repeating comment in many docker-compose.yaml files.

@GCHQDeveloper314 GCHQDeveloper314 added beginner tech debt Relates to Technical Debt Docker Issue related to the Docker side of the project labels Aug 11, 2022
@GCHQDeveloper314 GCHQDeveloper314 added this to the v2.0.0 milestone Aug 11, 2022
@lb324567 lb324567 modified the milestones: v2.0.0, post-v2.0_backlog Nov 9, 2022
@GCHQDeveloper314 GCHQDeveloper314 added good first issue Small, lower complexity and doesn't require pre-existing Gaffer knowledge and removed beginner labels Jul 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Docker Issue related to the Docker side of the project good first issue Small, lower complexity and doesn't require pre-existing Gaffer knowledge tech debt Relates to Technical Debt
Projects
None yet
Development

No branches or pull requests

2 participants