This project aims to address this challenge by providing a fluent, type-safe query builder for Spark SQL. Using this library, you can construct SQL queries in pure Scala code, without the need for string manipulation or manual query construction.
To get a local copy up and running follow these simple example steps.
To compile, run, test, and package this project: OpenJDK/JDK 8, 11 or 17, and sbt >= 1.0.0 are required.
-
Clone the repo:
$ git clone https://github.com/0zymandia5/spark_sql_query_builder.git
-
Navigate to the project directory:
$ cd ./spark_sql_query_builder
-
Start the sbt interactive shell:
$ sbt
- A message like this will show up:
[info] started sbt server
- A message like this will show up:
-
Compile the project:
sbt:spark_sql_query_builder> compile
- An output like this will show up:
[success] Total time: 4 s, completed Sep 28, 2024, 1:46:22 PM
- An output like this will show up:
-
Run the compiled project:
sbt:spark_sql_query_builder> run
- An output like this will show up:
[info] running com.spark.main.main SELECT geo AS GEO_NAME, mkt AS MARKET_NAME, COUNTRY_NAME FROM Hierarchy h [success] Total time: 0 s, completed Sep 28, 2024, 1:48:07 PM
- An output like this will show up:
-
To exit the sbt interactive shell:
sbt:spark_sql_query_builder> exit
Distributed under the MIT License. See LICENSE
for more information.