Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add view support for Hive catalog #8698

Closed
ajantha-bhat opened this issue Oct 2, 2023 · 9 comments
Closed

Add view support for Hive catalog #8698

ajantha-bhat opened this issue Oct 2, 2023 · 9 comments
Assignees
Labels

Comments

@ajantha-bhat
Copy link
Member

similar to #7913

@ajantha-bhat
Copy link
Member Author

@nk1506: Would you like to work on this?

@nk1506
Copy link
Contributor

nk1506 commented Oct 3, 2023

Yes, I would like to contribute on this. Thanks @ajantha-bhat

@nk1506
Copy link
Contributor

nk1506 commented Oct 19, 2023

As part of this I have couple of questions ?
For IcebergTable we use TableType as EXTERNAL_TABLE as well as properties with TABLE_TYPE_PROP.

With View Support:

  1. We can set TableType as VIRTUAL_VIEW
  2. It will make implementation more clean.
  3. We can also follow the strategy of managing extra field with table properties.
  4. To make backward compatibility we must add the required checks every where.
  5. IMetaStoreClient doesn't have specific APIs for Views.
  6. HiveTableOperations/HiveViewOperations must add all the required checks to manage same table/view name.

Please share your thoughts. CC: @ajantha-bhat , @jbonofre .
Also tag the relevant people.

@pvary
Copy link
Contributor

pvary commented Oct 19, 2023

@deniskuzZ: Is there there something like this ongoing in the Hive codebase?

@ajantha-bhat
Copy link
Member Author

ajantha-bhat commented Oct 19, 2023

Please share your thoughts. CC: @ajantha-bhat , @jbonofre .
Also tag the relevant people.

I think we can use VIRTUAL_VIEW type to avoid adding custom properties to the table.

@pvary:
Thanks for commenting. I was about to tag you.
Since this is a change for HiveCatalog (to extend BaseViewOperations), I think it should be done in this repo?
As per my knowledge, only the integration code with Hive 4.x is kept in Hive codebase.

@pvary
Copy link
Contributor

pvary commented Oct 19, 2023

@ajantha-bhat: The question is whether the Hive 3 HMS API is enough for the integration, or not. I would prefer if it would be enough, but we should definitely try to involve the Hive developers too.

@deniskuzZ
Copy link
Member

deniskuzZ commented Nov 9, 2023

@deniskuzZ: Is there there something like this ongoing in the Hive codebase?

hi @pvary, we have iceberg MV backed by iceberg table support in Hive where query definitions are stored in HMS and data in Iceberg table: HIVE-26578, HIVE-27101
Currently investigating how we could compute the delta between snapshots for incremental MV rebuild.

Copy link

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

@github-actions github-actions bot added the stale label Sep 22, 2024
@ajantha-bhat
Copy link
Member Author

Closed via #9852

@nastra nastra moved this from In progress to Done in View support Nov 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

No branches or pull requests

4 participants