Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(dataset): use sqlglot for DML check #31024

Merged
merged 2 commits into from
Nov 22, 2024
Merged

fix(dataset): use sqlglot for DML check #31024

merged 2 commits into from
Nov 22, 2024

Conversation

betodealmeida
Copy link
Member

SUMMARY

Use sqlglot when getting column metadata for a virtual dataset. We currently use sqlglot to check for DML in SQL Lab, but when we save a query as a dataset we're still using sqlparse.

Part of SIP-117.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A

TESTING INSTRUCTIONS

Run this query in SQL Lab:

with source as ( select 1 as one ) select * from source

This query is flagged incorrectly as DML by sqlparse, but not by sqlglot.

After running, save the query as a dataset. It should work, but currently it fails because sqlparse flags the query as having DML.

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@dosubot dosubot bot added the data:dataset Related to dataset configurations label Nov 21, 2024
@betodealmeida betodealmeida merged commit 832fed1 into master Nov 22, 2024
33 checks passed
asher-lab added a commit to asher-lab/superset that referenced this pull request Nov 22, 2024
@asher-lab @michael-s-molina
@sadpandajoe @mistercrunch @msyavuz @geido @villebro @PaoloTerzi @samarsrivastav @rusackas @amotl @LevisNgigi @dependabot @eschutho @theoriginalgri @lodu @betodealmeida @github-actions @actions-user @justinpark @dimbleby @jpchev @lindenh @seiyab
fix(import): Merge apache/master and resolve conflicts (#20)
8b446df 
* fix: Exception handling for SQL Lab views (apache#30897)

* fix: don't show metadata for embedded dashboards (apache#30875)

* feat: add logging durations for screenshot async service (apache#30884)

* refactor(Avatar): Migrate Avatar to Ant Design 5 (apache#30740)

* refactor(input): Migrate Input component to Ant Design 5 (apache#30730)

* fix(empty dashboards): Allow downloading a screenshot of an empty dashboard (apache#30767)

Co-authored-by: Geido <[email protected]>

* chore(ci): add tai and michael to helm owners (apache#30925)

* fix(helm): use submodule on helm release action (apache#30924)

* docs: add Free2Move to INTHEWILD.md (apache#30930)

* fix: blocks UI elements on right side (apache#30886)

Co-authored-by: Evan Rusackas <[email protected]>

* chore(deps): Migrate from `crate[sqlalchemy]` to `sqlalchemy-cratedb` (apache#29243)

* chore: update change log, UPDATING.md and bug-report.yml for 4.1 release (apache#30915)

* chore(docs): Update list of supported databases to include CrateDB (apache#30946)

* feat(trino,presto): add missing time grains (apache#30926)

* fix(Dashboard): Exclude edit param in async screenshot (apache#30962)

* fix(Card): Use correct class names for Ant Design 5 Card component (apache#30964)

* chore(Accessibility): Fix accessibility for 'Show x entries' dropdown in tables (apache#30818)

* build(deps): bump cross-spawn from 7.0.3 to 7.0.6 in /superset-frontend/cypress-base (apache#30969)

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* fix(release validation): scripts now support RSA and EDDSA keys. (apache#30967)

* build(deps): bump cross-spawn from 7.0.3 to 7.0.6 in /docs (apache#30970)

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* chore: add performance information to tooltip (apache#30948)

* fix(helm-chart): Fix broken PodDisruptionBudget due to introduction of extraLabels. (apache#30966)

* chore(helm): bumping app version to 4.1.1 in helm chart (apache#30918)

* chore: add unit tests for `is_mutating()` (apache#31021)

* chore(🦾): bump python numexpr 2.10.0 -> 2.10.1 (apache#31006)

Co-authored-by: GitHub Action <[email protected]>

* chore(🦾): bump python cffi 1.16.0 -> 1.17.1 (apache#31002)

Co-authored-by: GitHub Action <[email protected]>

* chore(🦾): bump python flask-babel subpackage(s) (apache#31000)

Co-authored-by: GitHub Action <[email protected]>

* fix(Dashboard): Retain colors when color scheme not set (apache#30646)

* fix(explore): verified props is not updated (apache#31008)

* chore: publish wheels (apache#30981)

* docs: Embedded sdk (apache#30972)

* fix(imports): import query_context for imports with charts (apache#30887)

* docs: Update doc about CSV upload (apache#30867)

Co-authored-by: Evan Rusackas <[email protected]>

* chore(🦾): bump python cattrs 23.2.3 -> 24.1.2 (apache#30998)

Co-authored-by: GitHub Action <[email protected]>

* fix(dataset): use sqlglot for DML check (apache#31024)

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Michael S. Molina <[email protected]>
Co-authored-by: Joe Li <[email protected]>
Co-authored-by: Maxime Beauchemin <[email protected]>
Co-authored-by: Mehmet Salih Yavuz <[email protected]>
Co-authored-by: Geido <[email protected]>
Co-authored-by: Ville Brofeldt <[email protected]>
Co-authored-by: Paolo Terzi <[email protected]>
Co-authored-by: Sukuna <[email protected]>
Co-authored-by: Evan Rusackas <[email protected]>
Co-authored-by: Andreas Motl <[email protected]>
Co-authored-by: Andreas Motl <[email protected]>
Co-authored-by: Levis Mbote <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Elizabeth Thompson <[email protected]>
Co-authored-by: Christoph Keller <[email protected]>
Co-authored-by: lodu <[email protected]>
Co-authored-by: Beto Dealmeida <[email protected]>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: GitHub Action <[email protected]>
Co-authored-by: JUST.in DO IT <[email protected]>
Co-authored-by: David Hotham <[email protected]>
Co-authored-by: Giampaolo Capelli <[email protected]>
Co-authored-by: Linden <[email protected]>
Co-authored-by: Seiya <[email protected]>
Co-authored-by: Asher Manangan <[email protected]>
@sadpandajoe sadpandajoe added the v4.1 Label added by the release manager to track PRs to be included in the 4.1 branch label Nov 22, 2024
@rusackas rusackas deleted the fix-dml-2 branch November 25, 2024 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@eschutho eschutho eschutho approved these changes

Assignees
No one assigned
Labels
data:dataset Related to dataset configurations size/M v4.1 Label added by the release manager to track PRs to be included in the 4.1 branch
Projects
Apache Superset 4.1.2
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@betodealmeida @eschutho @sadpandajoe