Skip to content

Commit

Permalink
[VL] Add gluten iceberg jar to bundle package (#6008)
Browse files Browse the repository at this point in the history
* Add iceberg to package

* update doc

* fix comment
  • Loading branch information
leoluan2009 authored Jun 11, 2024
1 parent 1d0bb52 commit d3bd3d7
Show file tree
Hide file tree
Showing 3 changed files with 11 additions and 6 deletions.
3 changes: 1 addition & 2 deletions docs/get-started/Velox.md
Original file line number Diff line number Diff line change
Expand Up @@ -303,8 +303,7 @@ First of all, compile gluten-iceberg module by a `iceberg` profile, as follows:
mvn clean package -Pbackends-velox -Pspark-3.3 -Piceberg -DskipTests
```

Then, put the additional `gluten-iceberg-XX-SNAPSHOT.jar` to the class path (usually it's `$SPARK_HOME/jars`).
The gluten-iceberg jar is in `gluten-iceberg/target` directory.
Once built successfully, iceberg features will be included in gluten-velox-bundle-X jar. Then you can query iceberg table by gluten/velox without scan's fallback.

After the two steps, you can query iceberg table by gluten/velox without scan's fallback.

Expand Down
4 changes: 0 additions & 4 deletions gluten-iceberg/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -63,10 +63,6 @@
<type>test-jar</type>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_${scala.binary.version}</artifactId>
</dependency>
<dependency>
<groupId>org.apache.spark</groupId>
<artifactId>spark-core_${scala.binary.version}</artifactId>
Expand Down
10 changes: 10 additions & 0 deletions package/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,16 @@
</dependency>
</dependencies>
</profile>
<profile>
<id>iceberg</id>
<dependencies>
<dependency>
<groupId>org.apache.gluten</groupId>
<artifactId>gluten-iceberg</artifactId>
<version>${project.version}</version>
</dependency>
</dependencies>
</profile>
</profiles>

<build>
Expand Down

0 comments on commit d3bd3d7

Please sign in to comment.