feat(helm-chart): remove helm-chart support #110

unknowntpo · 2024-11-25T00:23:31Z

Resolve #108

…log.sh

unknowntpo · 2024-11-27T10:00:14Z

@danhuawang @xunliu would you like to review this PR ?

playground.sh

tweak enable-ranger option Co-authored-by: JUN <[email protected]>

xunliu · 2024-12-02T05:01:42Z

@unknowntpo I will review this PR today, Thanks.

danhuawang · 2024-12-03T03:18:10Z

init/hive/init.sh

@@ -21,7 +21,6 @@
 sed -i '$d' /usr/local/sbin/start.sh
 sed -i '$d' /usr/local/sbin/start.sh
 cp /tmp/hive/core-site.xml /tmp/hadoop-conf
-sed -i "s|hdfs://localhost:9000|hdfs://${HIVE_HOST_IP}:9000|g" /usr/local/hive/conf/hive-site.xml
 /bin/bash /usr/local/sbin/start.sh


There's placeholder in the configuration file of the hive image: https://github.com/apache/gravitino/blob/main/dev/docker/hive/hive-site.xml

I think it needs to be replaced to hdfs://hive:9000

I think we don't need to add this line, because in start.sh, It has already used sed to replace it with $(hostname).

sed -i "s/__REPLACE__HOST_NAME/$(hostname)/g" ${HIVE_CONF_DIR}/hive-site.xml

https://github.com/apache/gravitino/blob/branch-0.7/dev/docker/hive/start.sh#L37

I think we can directly replace tail -f /dev/null with empty.
Because maybe tail -f /dev/null is not in the last line.

danhuawang · 2024-12-03T03:25:56Z

init/hive/init.sh

 sed -i '$d' /usr/local/sbin/start.sh
 sed -i '$d' /usr/local/sbin/start.sh


This is the original start.sh in image: https://github.com/apache/gravitino/blob/main/dev/docker/hive/start.sh
Is it necessary to remove the last two lines.

Is it necessary to remove the last two lines.

I don't get it. what do you mean ?

@xunliu @danhuawang fixed at ed6c5b5

I changed it to

sed -i -E 's/tail -f \/dev\/null/\s/g' /usr/local/sbin/start.sh

, since last line might be a blank string.

fixed at 7ed9dae

danhuawang · 2024-12-03T03:29:07Z

healthcheck/gravitino-healthcheck.sh

 while [ $attempt -lt $max_attempts ]; do
-  response=$(curl -X GET -H "Content-Type: application/json" http://${HOST_IP}:8090/api/version)
+  response=$(curl -X GET -H "Content-Type: application/json" http://gravitino:8090/api/version)


Is it 127.0.0.1 as the hostname inside the gravitino container?

yeah, reverted to 127.0.0.1 at f4404e6.

danhuawang · 2024-12-03T03:32:45Z

healthcheck/trino-healthcheck.sh

@@ -20,7 +20,7 @@
 set -ex

 # Because trino-connector must first synchronize a default metalake from the Gravitino server
-response=$(trino --server ${TRINO_HOST_IP}:8080 --execute "SHOW CATALOGS LIKE 'catalog_hive'")
+response=$(trino --server trino:8080 --execute "SHOW CATALOGS LIKE 'catalog_hive'")


Why not use localhost as hostname ?

sure, I changed this to localhost.
1640e8f

danhuawang · 2024-12-03T03:47:14Z

Do we still need the following code in init/spark/init.sh ?

# inject host information
sed -i 's/__GRAVITINO_HOST_IP__/'"$GRAVITINO_HOST_IP"'/g' /opt/spark/conf/spark-defaults.conf
sed -i 's/__HIVE_HOST_IP__/'"$HIVE_HOST_IP"'/g' /opt/spark/conf/spark-defaults.conf

danhuawang · 2024-12-03T03:53:12Z

Shall we remove the placeholder "MYSQL_HOST_IP","HIVE_HOST_IP" in init/gravitino/gravitino.conf ? If removed, set env var steps in init/gravitino/init.sh is also no longer needed.

unknowntpo · 2024-12-05T14:07:03Z

Do we still need the following code in init/spark/init.sh ?

# inject host information
sed -i 's/__GRAVITINO_HOST_IP__/'"$GRAVITINO_HOST_IP"'/g' /opt/spark/conf/spark-defaults.conf
sed -i 's/__HIVE_HOST_IP__/'"$HIVE_HOST_IP"'/g' /opt/spark/conf/spark-defaults.conf

ok, removed.

af41f3f

unknowntpo · 2024-12-05T14:07:21Z

Shall we remove the placeholder "MYSQL_HOST_IP","HIVE_HOST_IP" in init/gravitino/gravitino.conf ? If removed, set env var steps in init/gravitino/init.sh is also no longer needed.

ok, removed.

59e18da

… /dev/null

danhuawang · 2024-12-12T03:35:43Z

LGTM.
@xunliu

unknowntpo added 13 commits November 25, 2024 08:22

feat(helm-chart): remove helm-chart support

c3f3302

feat(playground.sh): revert helm related code

c7e41ea

feat(init/common/init_metalake_catalog.sh): revert init_metalake_cata…

9b8b8fc

…log.sh

feat(healthcheck/mysql-healthcheck.sh): revert

2353aae

feat(healthcheck/gravitino-healthcheck.sh): revert

1d1ea10

feat(healthcheck/hive-healthcheck.sh): revert

6c7a6f1

feat(healthcheck/trino-healthcheck.sh): revert

23e1e74

feat(docker-compose.yaml): revert

5fc913a

feat(init/jupyter/gravitino-trino-example.ipynb): revert

2d8a533

feat(init/jupyter/gravitino_llamaIndex_demo.ipynb): revert

f214024

feat(init/jupyter/gravitino-fileset-example.ipynb): revert

752bedb

feat(init/jupyter/gravitino-spark-trino-example.ipynb): revert

57a91ae

feat(README.md): revert

261e5eb

xunliu mentioned this pull request Nov 26, 2024

[#102] improvement: improve startup script #103

Draft

unknowntpo added 3 commits November 26, 2024 08:58

feat(playground.sh): revert helm related code

0e3b4c9

fix: revert init.sh

25c6507

refactor(README.md): revert helm-chart related doc

9faba33

unknowntpo marked this pull request as ready for review November 27, 2024 09:59

orenccl reviewed Nov 27, 2024

View reviewed changes

playground.sh Outdated Show resolved Hide resolved

unknowntpo and others added 2 commits November 27, 2024 18:48

Update playground.sh

1262b21

tweak enable-ranger option Co-authored-by: JUN <[email protected]>

refactor(playground.sh): tweak --enable-ranger option

353d1cf

xunliu assigned unknowntpo Dec 2, 2024

danhuawang reviewed Dec 3, 2024

View reviewed changes

unknowntpo added 6 commits December 4, 2024 08:11

refactor(hive/init.sh): should use sed to replace __REPLACE__HOST_NAME

e8b70a3

refactor(healthcheck/trino-healthcheck.sh): use localhost as hostname

1640e8f

refactor(init/spark/init.sh): no need to substitute host ip info

af41f3f

refactor(init/gravitino/gravitino.conf): hard code docker service name

59e18da

refactor(healthcheck/gravitino-healthcheck.sh): use 127.0.0.1 as host ip

f4404e6

refactor(hive/init.sh): no need to replace hive host name

badb490

unknowntpo force-pushed the feat-remove-helm-chart branch from 0367525 to badb490 Compare December 5, 2024 14:05

unknowntpo added 2 commits December 10, 2024 08:31

refactor(hive/init.sh): delete last line of start.sh if it is tail -f…

ed6c5b5

… /dev/null

refactor(init/hive): use sed to replace tail -f command

7ed9dae

unknowntpo force-pushed the feat-remove-helm-chart branch from 1b20f74 to 7ed9dae Compare December 11, 2024 00:45

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(helm-chart): remove helm-chart support #110

feat(helm-chart): remove helm-chart support #110

unknowntpo commented Nov 25, 2024

unknowntpo commented Nov 27, 2024

xunliu commented Dec 2, 2024

danhuawang Dec 3, 2024

unknowntpo Dec 4, 2024

xunliu Dec 10, 2024

danhuawang Dec 3, 2024

unknowntpo Dec 4, 2024

unknowntpo Dec 10, 2024

unknowntpo Dec 11, 2024

danhuawang Dec 3, 2024

unknowntpo Dec 5, 2024 •

edited

Loading

danhuawang Dec 3, 2024

unknowntpo Dec 5, 2024 •

edited

Loading

danhuawang commented Dec 3, 2024

danhuawang commented Dec 3, 2024

unknowntpo commented Dec 5, 2024 •

edited

Loading

unknowntpo commented Dec 5, 2024 •

edited

Loading

danhuawang commented Dec 12, 2024

		sed -i '$d' /usr/local/sbin/start.sh
		sed -i '$d' /usr/local/sbin/start.sh

feat(helm-chart): remove helm-chart support #110

Are you sure you want to change the base?

feat(helm-chart): remove helm-chart support #110

Conversation

unknowntpo commented Nov 25, 2024

unknowntpo commented Nov 27, 2024

xunliu commented Dec 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

unknowntpo Dec 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

unknowntpo Dec 5, 2024 • edited Loading

Choose a reason for hiding this comment

danhuawang commented Dec 3, 2024

danhuawang commented Dec 3, 2024

unknowntpo commented Dec 5, 2024 • edited Loading

unknowntpo commented Dec 5, 2024 • edited Loading

danhuawang commented Dec 12, 2024

unknowntpo Dec 5, 2024 •

edited

Loading

unknowntpo Dec 5, 2024 •

edited

Loading

unknowntpo commented Dec 5, 2024 •

edited

Loading

unknowntpo commented Dec 5, 2024 •

edited

Loading