Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SNOW-1038770: Failed to initialize MemoryUtil. Was Java started with --add-opens=java.base/java.nio=ALL-UNNAMED? #82

Open
knanne opened this issue Feb 1, 2024 · 3 comments
Open

SNOW-1038770: Failed to initialize MemoryUtil. Was Java started with --add-opens=java.base/java.nio=ALL-UNNAMED? #82

knanne opened this issue Feb 1, 2024 · 3 comments
Assignees
@sfc-gh-jfreeberg

Comments

@knanne
Copy link

knanne commented Feb 1, 2024

Trying to bump our our Springboot application to Java 17, was excited to see 17 support in new release of snowpark 1.10

However, we are still getting the below error message. The error happens immediately after performing an INSERT statement on the database successfully. The code doing that insert is

...
df_log.write().option("columnOrder", "name").mode(SaveMode.Append).saveAsTable("AUDIT_JOB_LOGS");

session.close();

It looks like the release included passing of some java args 8b4c579 as described as a "short-term solution" in https://community.snowflake.com/s/article/JDBC-Driver-Compatibility-Issue-With-JDK-16-and-Later

Is this maybe not excluding something important?

My build.gradle file is also below

error

[restartedMain] INFO com.snowflake.snowpark.Session - Closing stderr and redirecting to stdout
[restartedMain] INFO com.snowflake.snowpark.Session - Done closing stderr and redirecting to stdout
[restartedMain] INFO com.snowflake.snowpark.internal.ParameterUtils - set JDBC client memory limit to 10240
[restartedMain] INFO com.snowflake.snowpark.Session - Snowpark Session information: {
 "snowpark.version" : "1.10.0",
 "client.language": "Java",
 "java.version" : "17.0.9",
 "scala.version" : "2.12.18",
 "jdbc.session.id" : "66284275835526",
 "os.name" : "Windows 10",
 "jdbc.version" : "3.14.4",
 "snowpark.library" : "/C:/Users/kain.nanne/.gradle/caches/modules-2/files-2.1/com.snowflake/snowpark/1.10.0/c13dc54ec6cdd0e2468d7f3ec48ccc68ffbd2e7a/snowpark-1.10.0.jar",
 "scala.library" : "/C:/Users/kain.nanne/.gradle/caches/modules-2/files-2.1/org.scala-lang/scala-library/2.12.18/fa825f984172f6801e3d8c476071b1823c0a925b/scala-library-2.12.18.jar",
 "jdbc.library" : "/C:/Users/kain.nanne/.gradle/caches/modules-2/files-2.1/net.snowflake/snowflake-jdbc/3.14.4/aa39560c63fc8a578aecf6a0314d3cc7e0ae507/snowflake-jdbc-3.14.4.jar"
}
[restartedMain] INFO com.snowflake.snowpark.internal.ServerConnection - Actively querying parameter QUERY_TAG from server.
[restartedMain] INFO com.snowflake.snowpark.internal.ServerConnection - Execute query [queryID: 01b21079-0000-b063-0000-3c491284b042] DESCRIBE TABLE AUDIT_JOB_LOGS
[restartedMain] INFO com.snowflake.snowpark.internal.ServerConnection - Execute query [queryID: 01b21079-0000-b07d-0000-3c491284ad7e] DESCRIBE TABLE AUDIT_JOB_LOGS
[restartedMain] INFO com.snowflake.snowpark.internal.ServerConnection - Execute query [queryID: 01b21079-0000-b063-0000-3c491284b082]  INSERT  INTO AUDIT_JOB_LOGS SELECT  *  FROM ( SELECT "JOB_ID", "SERVER", "JOB_TYPE", "START_TIME", "END_TIME", "ARGS", "RECORDS", "DURATION_SEC", "STATUS", "ERROR", "COMMENT" FROM ( SELECT "JOB_ID", "SERVER", "JOB_TYPE", "START_TIME", to_timestamp("END_TIME") AS "END_TIME", "ARGS", "RECORDS", "DURATION_SEC", "STATUS", "ERROR", "COMMENT" FROM ( SELECT  *  FROM ( VALUES ('5bdcf6c1-59bd-4655-adad-540081387340', 'AVD0629328-0: kain.nanne', 'JAVA APP STARTUP', NULL :: string, '2024-02-01 12:01:10', NULL :: string, NULL :: int, NULL :: int, 'COMPLETE', NULL :: string, 'LOG: INFO | BATCH SIZE: 100000 | SERVER TIME: 2024-02-01 12:01:10 | NEXT SCHEDULE: 2024-02-02 06:00:00') AS SNOWPARK_TEMP_TABLE_ON89TG4BKZYLF13("JOB_ID", "SERVER", "JOB_TYPE", "START_TIME", "END_TIME", "ARGS", "RECORDS", "DURATION_SEC", "STATUS", "ERROR", "COMMENT")))))
[restartedMain] INFO com.snowflake.snowpark.Session - Closing session: {
 "snowpark.version" : "1.10.0",
 "client.language": "Java",
 "java.version" : "17.0.9",
 "scala.version" : "2.12.18",
 "jdbc.session.id" : "66284275835526",
 "os.name" : "Windows 10",
 "jdbc.version" : "3.14.4",
 "snowpark.library" : "/C:/Users/kain.nanne/.gradle/caches/modules-2/files-2.1/com.snowflake/snowpark/1.10.0/c13dc54ec6cdd0e2468d7f3ec48ccc68ffbd2e7a/snowpark-1.10.0.jar",
 "scala.library" : "/C:/Users/kain.nanne/.gradle/caches/modules-2/files-2.1/org.scala-lang/scala-library/2.12.18/fa825f984172f6801e3d8c476071b1823c0a925b/scala-library-2.12.18.jar",
 "jdbc.library" : "/C:/Users/kain.nanne/.gradle/caches/modules-2/files-2.1/net.snowflake/snowflake-jdbc/3.14.4/aa39560c63fc8a578aecf6a0314d3cc7e0ae507/snowflake-jdbc-3.14.4.jar"
}
[restartedMain] INFO com.snowflake.snowpark.Session - Canceling all running query
java.lang.RuntimeException: Failed to initialize MemoryUtil. Was Java started with `--add-opens=java.base/java.nio=ALL-UNNAMED`? (See https://arrow.apache.org/docs/java/install.html)
        at net.snowflake.client.jdbc.internal.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:146)
        at net.snowflake.client.jdbc.internal.apache.arrow.memory.ArrowBuf.getDirectBuffer(ArrowBuf.java:231)
        at net.snowflake.client.jdbc.internal.apache.arrow.memory.ArrowBuf.nioBuffer(ArrowBuf.java:226)
        at net.snowflake.client.jdbc.internal.apache.arrow.vector.ipc.ReadChannel.readFully(ReadChannel.java:87)
        at net.snowflake.client.jdbc.internal.apache.arrow.vector.ipc.message.MessageSerializer.readMessageBody(MessageSerializer.java:728)
        at net.snowflake.client.jdbc.internal.apache.arrow.vector.ipc.message.MessageChannelReader.readNext(MessageChannelReader.java:67)
        at net.snowflake.client.jdbc.internal.apache.arrow.vector.ipc.ArrowStreamReader.loadNextBatch(ArrowStreamReader.java:145)
        at net.snowflake.client.jdbc.SnowflakeResultSetSerializableV1.setFirstChunkRowCountForArrow(SnowflakeResultSetSerializableV1.java:1127)
        at net.snowflake.client.jdbc.SnowflakeResultSetSerializableV1.create(SnowflakeResultSetSerializableV1.java:601)
        at net.snowflake.client.jdbc.SnowflakeResultSetSerializableV1.create(SnowflakeResultSetSerializableV1.java:497)
        at net.snowflake.client.core.SFResultSetFactory.getResultSet(SFResultSetFactory.java:34)
        at net.snowflake.client.core.SFStatement.executeQueryInternal(SFStatement.java:234)
        at net.snowflake.client.core.SFStatement.executeQuery(SFStatement.java:140)
        at net.snowflake.client.core.SFStatement.execute(SFStatement.java:776)
        at net.snowflake.client.core.SFStatement.execute(SFStatement.java:684)
        at net.snowflake.client.jdbc.SnowflakeStatementV1.executeQueryInternal(SnowflakeStatementV1.java:267)
        at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.executeQuery(SnowflakePreparedStatementV1.java:122)
        at com.snowflake.snowpark.internal.ServerConnection.$anonfun$runQueryGetResult$1(ServerConnection.scala:388)
        at com.snowflake.snowpark.internal.ServerConnection.withValidConnection(ServerConnection.scala:925)
        at com.snowflake.snowpark.internal.ServerConnection.runQueryGetResult(ServerConnection.scala:383)
        at com.snowflake.snowpark.internal.ServerConnection.runQuery(ServerConnection.scala:359)
        at com.snowflake.snowpark.Session.cancelAll(Session.scala:154)
        at com.snowflake.snowpark.Session.close(Session.scala:1243)
        at com.snowflake.snowpark_java.Session.close(Session.java:484)
        at fhirtransformer.engine.SnowflakeConfig.logJavaJobStarted(SnowflakeConfig.java:153)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:365)
        at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:237)
        at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:168)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:178)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:171)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:149)
        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:451)
        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:384)
        at org.springframework.boot.context.event.EventPublishingRunListener.ready(EventPublishingRunListener.java:109)
        at org.springframework.boot.SpringApplicationRunListeners.lambda$ready$6(SpringApplicationRunListeners.java:80)
        at java.base/java.lang.Iterable.forEach(Iterable.java:75)
        at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118)
        at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112)
        at org.springframework.boot.SpringApplicationRunListeners.ready(SpringApplicationRunListeners.java:80)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:352)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1358)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1347)
        at fhirtransformer.StreamApp.main(StreamApp.java:12)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:50)
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field long java.nio.Buffer.address accessible: module java.base does not "opens java.nio" to unnamed module @4617c264
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
        at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
        at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:178)
        at java.base/java.lang.reflect.Field.setAccessible(Field.java:172)
        at net.snowflake.client.jdbc.internal.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:91)
        ... 51 more
[restartedMain] ERROR com.snowflake.snowpark.internal.ServerConnection - failed to execute query:
select system$cancel_all_queries(66284275835526)
[restartedMain] ERROR org.springframework.boot.SpringApplication - Application run failed
com.snowflake.snowpark.SnowparkClientException: Error Code: 0415, Error message: Failed to close this session. The error is: JDBC driver internal error: exception creating result java.lang.ExceptionInInitializerError at net.snowflake.client.jdbc.internal.apache.arrow.memory.ArrowBuf.getDirectBuffer(ArrowBuf.java:231).
        at com.snowflake.snowpark.internal.ErrorMessage$.createException(ErrorMessage.scala:420)
        at com.snowflake.snowpark.internal.ErrorMessage$.MISC_FAILED_CLOSE_SESSION(ErrorMessage.scala:379)
        at com.snowflake.snowpark.Session.close(Session.scala:1247)
        at com.snowflake.snowpark_java.Session.close(Session.java:484)
        at fhirtransformer.engine.SnowflakeConfig.logJavaJobStarted(SnowflakeConfig.java:153)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.context.event.ApplicationListenerMethodAdapter.doInvoke(ApplicationListenerMethodAdapter.java:365)
        at org.springframework.context.event.ApplicationListenerMethodAdapter.processEvent(ApplicationListenerMethodAdapter.java:237)
        at org.springframework.context.event.ApplicationListenerMethodAdapter.onApplicationEvent(ApplicationListenerMethodAdapter.java:168)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:178)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:171)
        at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:149)
        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:451)
        at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:384)
        at org.springframework.boot.context.event.EventPublishingRunListener.ready(EventPublishingRunListener.java:109)
        at org.springframework.boot.SpringApplicationRunListeners.lambda$ready$6(SpringApplicationRunListeners.java:80)
        at java.base/java.lang.Iterable.forEach(Iterable.java:75)
        at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:118)
        at org.springframework.boot.SpringApplicationRunListeners.doWithListeners(SpringApplicationRunListeners.java:112)
        at org.springframework.boot.SpringApplicationRunListeners.ready(SpringApplicationRunListeners.java:80)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:352)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1358)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1347)
        at fhirtransformer.StreamApp.main(StreamApp.java:12)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:50)

build.gradle

plugins {
    id 'java'
    id 'java-library'
    id 'org.springframework.boot' version '3.2.1'
    id 'org.sonarqube' version '4.2.1.3168'
    id 'jacoco'
    id 'io.freefair.lombok' version '8.4'
}

apply plugin: 'io.spring.dependency-management'

group = 'fhirtransformer'
version = '1.0'
description = 'matchboxtransformer'
sourceCompatibility = 17

repositories {
    mavenCentral()
}

dependencies {
    developmentOnly 'org.springframework.boot:spring-boot-devtools'
    implementation 'org.springframework.boot:spring-boot'
    implementation 'org.springframework.boot:spring-boot-starter-web'
    implementation 'org.springframework.boot:spring-boot-autoconfigure'
    implementation 'org.springframework.boot:spring-boot-starter-actuator'
    implementation 'org.json:json:20230618'
    implementation 'org.javatuples:javatuples:1.2'
    testImplementation 'junit:junit:4.11'
    testImplementation 'org.springframework.boot:spring-boot-starter-test'
    testImplementation 'org.awaitility:awaitility:3.1.2'
    api 'com.snowflake:snowpark:1.10.0'
    api 'health.matchbox:matchbox-engine:3.3.3'
}

configurations.all {
   exclude group: 'ch.qos.logback', module: 'logback-core'
}

test {
    useJUnitPlatform()
}

jar {
    manifest {
        attributes 'Main-Class': 'fhirtransformer.StreamApp'
    }
}
@github-actions github-actions bot changed the title Failed to initialize MemoryUtil. Was Java started with --add-opens=java.base/java.nio=ALL-UNNAMED? SNOW-1038770: Failed to initialize MemoryUtil. Was Java started with --add-opens=java.base/java.nio=ALL-UNNAMED? Feb 1, 2024
@knanne
Copy link
Author

knanne commented Feb 1, 2024

Interesting enough, if I remove session.close(); the error is not thrown

However, when performing a SELECT later on, a similar error comes back.

Specifically this code block (where limitNum = 10)

log.info("Querying Snowflake");
DataFrame df_src = session.table(sourceTable).filter(Functions.col("TRANSFORMED_IND").equal_to(Functions.lit(0)));
if (limitNum>0) {
    log.info("Limiting Query to "+limitNum+" records");
    df_src = df_src.limit(limitNum);
}
long records_num = df_src.count();

fails with, error

[http-nio-8080-exec-2] INFO fhirtransformer.engine.Transformer - Querying Snowflake
[http-nio-8080-exec-2] INFO fhirtransformer.engine.Transformer - Limiting Query to 10 records
[2024-02-01 16:22:01.817] - 9660 SEVERE [http-nio-8080-exec-2] --- net.snowflake.client.core.SFStatement: Exception creating result
java.lang.NoClassDefFoundError: Could not initialize class net.snowflake.client.jdbc.internal.apache.arrow.memory.util.MemoryUtil
        at net.snowflake.client.jdbc.internal.apache.arrow.memory.ArrowBuf.getDirectBuffer(ArrowBuf.java:231)
        at net.snowflake.client.jdbc.internal.apache.arrow.memory.ArrowBuf.nioBuffer(ArrowBuf.java:226)
        at net.snowflake.client.jdbc.internal.apache.arrow.vector.ipc.ReadChannel.readFully(ReadChannel.java:87)
        at net.snowflake.client.jdbc.internal.apache.arrow.vector.ipc.message.MessageSerializer.readMessageBody(MessageSerializer.java:728)
        at net.snowflake.client.jdbc.internal.apache.arrow.vector.ipc.message.MessageChannelReader.readNext(MessageChannelReader.java:67)
        at net.snowflake.client.jdbc.internal.apache.arrow.vector.ipc.ArrowStreamReader.loadNextBatch(ArrowStreamReader.java:145)
        at net.snowflake.client.jdbc.SnowflakeResultSetSerializableV1.setFirstChunkRowCountForArrow(SnowflakeResultSetSerializableV1.java:1127)
        at net.snowflake.client.jdbc.SnowflakeResultSetSerializableV1.create(SnowflakeResultSetSerializableV1.java:601)
        at net.snowflake.client.jdbc.SnowflakeResultSetSerializableV1.create(SnowflakeResultSetSerializableV1.java:497)
        at net.snowflake.client.core.SFResultSetFactory.getResultSet(SFResultSetFactory.java:34)
        at net.snowflake.client.core.SFStatement.executeQueryInternal(SFStatement.java:234)
        at net.snowflake.client.core.SFStatement.executeQuery(SFStatement.java:140)
        at net.snowflake.client.core.SFStatement.execute(SFStatement.java:776)
        at net.snowflake.client.core.SFStatement.execute(SFStatement.java:684)
        at net.snowflake.client.jdbc.SnowflakeStatementV1.executeQueryInternal(SnowflakeStatementV1.java:267)
        at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.executeQuery(SnowflakePreparedStatementV1.java:122)
        at com.snowflake.snowpark.internal.ServerConnection.$anonfun$runQueryGetResult$1(ServerConnection.scala:388)
        at com.snowflake.snowpark.internal.ServerConnection.withValidConnection(ServerConnection.scala:925)
        at com.snowflake.snowpark.internal.ServerConnection.runQueryGetResult(ServerConnection.scala:383)
        at com.snowflake.snowpark.internal.analyzer.Query.runQueryGetResult(SnowflakePlan.scala:798)
        at com.snowflake.snowpark.internal.ServerConnection.$anonfun$executePlanInternal$2(ServerConnection.scala:750)
        at com.snowflake.snowpark.internal.analyzer.SnowflakePlan$.wrapException(SnowflakePlan.scala:159)
        at com.snowflake.snowpark.internal.ServerConnection.$anonfun$executePlanInternal$1(ServerConnection.scala:723)
        at com.snowflake.snowpark.internal.ServerConnection.withValidConnection(ServerConnection.scala:925)
        at com.snowflake.snowpark.internal.ServerConnection.executePlanInternal(ServerConnection.scala:723)
        at com.snowflake.snowpark.internal.ServerConnection.$anonfun$execute$1(ServerConnection.scala:689)
        at com.snowflake.snowpark.internal.ServerConnection.withValidConnection(ServerConnection.scala:925)
        at com.snowflake.snowpark.internal.ServerConnection.execute(ServerConnection.scala:689)
        at com.snowflake.snowpark.DataFrame.collect(DataFrame.scala:2225)
        at com.snowflake.snowpark.DataFrame.count(DataFrame.scala:2253)
        at com.snowflake.snowpark_java.DataFrame.count(DataFrame.java:791)
        at fhirtransformer.engine.Transformer.transform(Transformer.java:1387)
        at fhirtransformer.APIController.TransformEndpoint(APIController.java:63)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:262)
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:190)
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:917)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:829)
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:979)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014)
        at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:914)
        at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:590)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885)
        at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:205)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:109)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:340)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.RuntimeException: Failed to initialize MemoryUtil. Was Java started with `--add-opens=java.base/java.nio=ALL-UNNAMED`? (See https://arrow.apache.org/docs/java/install.html) [in thread "http-nio-8080-exec-5"]
        at net.snowflake.client.jdbc.internal.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:146)
        ... 87 more

[http-nio-8080-exec-2] ERROR com.snowflake.snowpark.internal.ServerConnection - failed to execute query:
 SELECT count(1) AS "COUNT(LITERAL())" FROM ( SELECT  *  FROM ( SELECT  *  FROM ( SELECT  *  FROM (SOURCE_RESOURCE)) WHERE ("TRANSFORMED_IND" = 0 :: int)) LIMIT 10) LIMIT 1
[2024-02-01 16:22:01.858] - 9660 SEVERE [http-nio-8080-exec-2] --- org.apache.catalina.core.ContainerBase.[Tomcat].[localhost].[/].[dispatcherServlet]: Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed: net.snowflake.client.jdbc.SnowflakeSQLException: JDBC driver internal error: exception creating result java.lang.NoClassDefFoundError: Could not initialize class net.snowflake.client.jdbc.internal.apache.arrow.memory.util.MemoryUtil at net.snowflake.client.jdbc.internal.apache.arrow.memory.ArrowBuf.getDirectBuffer(ArrowBuf.java:231).] with root cause
net.snowflake.client.jdbc.SnowflakeSQLException: JDBC driver internal error: exception creating result java.lang.NoClassDefFoundError: Could not initialize class net.snowflake.client.jdbc.internal.apache.arrow.memory.util.MemoryUtil at net.snowflake.client.jdbc.internal.apache.arrow.memory.ArrowBuf.getDirectBuffer(ArrowBuf.java:231).
        at net.snowflake.client.jdbc.SnowflakeStatementV1.executeQueryInternal(SnowflakeStatementV1.java:276)
        at net.snowflake.client.jdbc.SnowflakePreparedStatementV1.executeQuery(SnowflakePreparedStatementV1.java:122)
        at com.snowflake.snowpark.internal.ServerConnection.$anonfun$runQueryGetResult$1(ServerConnection.scala:388)
        at com.snowflake.snowpark.internal.ServerConnection.withValidConnection(ServerConnection.scala:925)
        at com.snowflake.snowpark.internal.ServerConnection.runQueryGetResult(ServerConnection.scala:383)
        at com.snowflake.snowpark.internal.analyzer.Query.runQueryGetResult(SnowflakePlan.scala:798)
        at com.snowflake.snowpark.internal.ServerConnection.$anonfun$executePlanInternal$2(ServerConnection.scala:750)
        at com.snowflake.snowpark.internal.analyzer.SnowflakePlan$.wrapException(SnowflakePlan.scala:159)
        at com.snowflake.snowpark.internal.ServerConnection.$anonfun$executePlanInternal$1(ServerConnection.scala:723)
        at com.snowflake.snowpark.internal.ServerConnection.withValidConnection(ServerConnection.scala:925)
        at com.snowflake.snowpark.internal.ServerConnection.executePlanInternal(ServerConnection.scala:723)
        at com.snowflake.snowpark.internal.ServerConnection.$anonfun$execute$1(ServerConnection.scala:689)
        at com.snowflake.snowpark.internal.ServerConnection.withValidConnection(ServerConnection.scala:925)
        at com.snowflake.snowpark.internal.ServerConnection.execute(ServerConnection.scala:689)
        at com.snowflake.snowpark.DataFrame.collect(DataFrame.scala:2225)
        at com.snowflake.snowpark.DataFrame.count(DataFrame.scala:2253)
        at com.snowflake.snowpark_java.DataFrame.count(DataFrame.java:791)
        at fhirtransformer.engine.Transformer.transform(Transformer.java:1387)
        at fhirtransformer.APIController.TransformEndpoint(APIController.java:63)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:262)
        at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:190)
        at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:917)
        at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:829)
        at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
        at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:979)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014)
        at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:914)
        at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:590)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885)
        at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:205)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:109)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
        at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:174)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:149)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:482)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:340)
        at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:391)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:896)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1744)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.base/java.lang.Thread.run(Thread.java:840)

@sfc-gh-bli
Copy link
Collaborator

It is a JDBC limitation. Snowflake JDBC replies dependencies such as Apache Arrow, which require those Java libs.
https://arrow.apache.org/docs/java/install.html#id3

@sfc-gh-jfreeberg
Copy link
Collaborator

@knanne -- Sorry for the late reply. Can you set --add-opens=java.base/java.nio=ALL-UNNAMED as described in this doc: https://docs.snowflake.com/en/developer-guide/snowpark/java/troubleshooting#unnamed-module-error-on-java-17

As Bing mentioned, Apache Arrow requires this parameter.

@sfc-gh-jfreeberg sfc-gh-jfreeberg self-assigned this Mar 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sfc-gh-jfreeberg sfc-gh-jfreeberg

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@knanne @sfc-gh-bli @sfc-gh-jfreeberg