Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No access to glue #2

Open
parisni opened this issue Aug 9, 2021 · 1 comment
Open

No access to glue #2

parisni opened this issue Aug 9, 2021 · 1 comment

Comments

@parisni
Copy link

parisni commented Aug 9, 2021

is this supposed to access to aws glue catalog ?
I get those warning together with an empty local metastore.

I made working spark 2.4.x + hive 1.2.1. Sadly no success with spark3 and your work.

scala> spark.sql("show databases").show
21/08/09 15:44:47 WARN HiveConf: HiveConf of name hive.metastore.client.factory.class does not exist
21/08/09 15:44:48 WARN HiveConf: HiveConf of name hive.stats.jdbc.timeout does not exist
21/08/09 15:44:48 WARN HiveConf: HiveConf of name hive.metastore.client.factory.class does not exist
21/08/09 15:44:48 WARN HiveConf: HiveConf of name hive.stats.retries.wait does not exist
21/08/09 15:44:49 WARN HiveConf: HiveConf of name hive.metastore.client.factory.class does not exist
21/08/09 15:44:53 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 2.3.0
21/08/09 15:44:53 WARN ObjectStore: setMetaStoreSchemaVersion called but recording version is disabled: version = 2.3.0, comment
+---------+
|namespace|
+---------+
|  default|
+---------+

@parisni
Copy link
Author

parisni commented Aug 10, 2021

I finally got this working by using this https://github.com/kuhnen/spark-glue/tree/to_docker

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant