Skip to content

Commit

Permalink
Fix potential double unload of JNI libraries on Java 11+
Browse files Browse the repository at this point in the history
Previously, the addHookForLibUnloading method added a shutdown hook
unconditionally to unload JNI libraries. This could potentially
lead to double unload issues on Java 11 and later versions where
the JVM automatically handles JNI library cleanup.

This commit modifies addHookForLibUnloading to conditionally add
the unload hook based on the JVM version:
- For Java 11 and later, the method adds a no-op hook since the JVM
  handles the cleanup automatically.
- For Java versions prior to 11, the method adds the provided hook
  to ensure proper cleanup.

This change ensures compatibility and prevents potential double
unload issues on Java 11 and later.
  • Loading branch information
yangzhg committed Jun 21, 2024
1 parent 88394d1 commit 91dbebc
Showing 1 changed file with 15 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -16,14 +16,28 @@
*/
package org.apache.spark.util

import org.apache.gluten.backendsapi.BackendsApiManager

import org.apache.commons.lang3.{JavaVersion, SystemUtils}

object GlutenShutdownManager {

def addHook(hook: () => Unit): AnyRef = {
ShutdownHookManager.addShutdownHook(ShutdownHookManager.DEFAULT_SHUTDOWN_PRIORITY)(hook)
}

def addHookForLibUnloading(hook: () => Unit): AnyRef = {
ShutdownHookManager.addShutdownHook(ShutdownHookManager.SPARK_CONTEXT_SHUTDOWN_PRIORITY)(hook)
ShutdownHookManager.addShutdownHook(ShutdownHookManager.SPARK_CONTEXT_SHUTDOWN_PRIORITY) {
if (
BackendsApiManager.getBackendName == "velox" && SystemUtils.isJavaVersionAtLeast(
JavaVersion.JAVA_11)
) {
// In Java 11 and later, the JVM automatically handles the cleanup of JNI libraries
() => Unit
} else {
hook
}
}
}

def addHookForTempDirRemoval(hook: () => Unit): AnyRef = {
Expand Down

0 comments on commit 91dbebc

Please sign in to comment.