added ports 4041-3 to docker compose so spark web ui can be accessed

spatialx-project · Apr 4, 2023 · 276af39 · 276af39
1 parent 2f6cbc9
commit 276af39
Show file tree

Hide file tree

Showing 2 changed files with 5 additions and 0 deletions.
diff --git a/README.md b/README.md
@@ -20,3 +20,5 @@ Open [http://localhost:8888](http://localhost:8888) in browser, there are 2 note
 - [geolake-scala-demo](spark/notebooks/geolake-scala-demo.ipynb): shows an example of how to use geolake scala api to read/write data from/to geolake.
 
 - [benchmark-portotaxi](spark/notebooks/benchmark-portotaxi.ipynb): this notebook runs a benchmark on portotaxi dataset which has 1.7M records. You will see the reading/writing performance of the 3 Parquet format(GeoLake Parqeut, GeoParquet, GeoParquet(bbox)) for spatial data. You will also see how the partition reolution parameter affects the performance.
+
+Ports 4041, 4042 and 4043 are also forwarded, so you can access Spark Web UI if necessary.
diff --git a/docker-compose.yml b/docker-compose.yml
@@ -15,6 +15,9 @@ services:
       - PYSPARK_PYTON=/usr/bin/python3.9
       - PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/spark/bin
     ports:
+      - 4041:4041
+      - 4042:4042
+      - 4043:4043
       - 8888:8888
       - 8080:8080
       - 18080:18080
Original file line number	Diff line number	Diff line change
Expand Up		@@ -20,3 +20,5 @@ Open [http://localhost:8888](http://localhost:8888) in browser, there are 2 note
		- [geolake-scala-demo](spark/notebooks/geolake-scala-demo.ipynb): shows an example of how to use geolake scala api to read/write data from/to geolake.

		- [benchmark-portotaxi](spark/notebooks/benchmark-portotaxi.ipynb): this notebook runs a benchmark on portotaxi dataset which has 1.7M records. You will see the reading/writing performance of the 3 Parquet format(GeoLake Parqeut, GeoParquet, GeoParquet(bbox)) for spatial data. You will also see how the partition reolution parameter affects the performance.

		Ports 4041, 4042 and 4043 are also forwarded, so you can access Spark Web UI if necessary.