[WIP] Introduce the "parser" feature to gate the SQL text processing and leaving only AST and other support types

[felipecrv]

This crate can become a very lightweight dependency to datafusion (and other projects) that have their own SQL parser but need to use datafusion-sqlparser-rs AST types to interact with the rest of datafusion (as a modular library and not as a full-featured SQL engine).

WORK-IN-PROGRESS notice: this is draft state and I want to see how CI behaves, reduce the number of cfg(feature = "parser") annotations needed and ensure datafusion still builds even if the "parser" feature is disabled.

Before merging, "parser" will of course, become part of the default set of features of the crate.

[alamb]

This crate can become a very lightweight dependency to datafusion (and other projects) that have their own SQL parser but need to use datafusion-sqlparser-rs AST types to interact with the rest of datafusion (as a modular library and not as a full-featured SQL engine).

BTW another way to approach this issue would be to remove the dependency of sqlparser from DataFusion (e.g. make the parsing an optional module)

[felipecrv]

This crate can become a very lightweight dependency to datafusion (and other projects) that have their own SQL parser but need to use datafusion-sqlparser-rs AST types to interact with the rest of datafusion (as a modular library and not as a full-featured SQL engine).

BTW another way to approach this issue would be to remove the dependency of sqlparser from DataFusion (e.g. make the parsing an optional module)

I tried, but there are non-trivial dependencies on the ast:: and dynamic dispatch to the dialect trait. I think it's reasonable to share the ast:: types instead of abstracting them, hence my idea of disabling the parsing from the crate and leave the rest available.

[alamb]

I tried, but there are non-trivial dependencies on the ast:: and dynamic dispatch to the dialect trait. I think it's reasonable to share the ast:: types instead of abstracting them, hence my idea of disabling the parsing from the crate and leave the rest available.

The dependencies on the ast can be handled by copying the definitions into DataFusion

The dpeendencies on Dialect is maybe harder to imagine

BTW to be clear the idea in this PR seems fine to me -- I just figured I would comment

[felipecrv]

I tried, but there are non-trivial dependencies on the ast:: and dynamic dispatch to the dialect trait. I think it's reasonable to share the ast:: types instead of abstracting them, hence my idea of disabling the parsing from the crate and leave the rest available.

The dependencies on the ast can be handled by copying the definitions into DataFusion

But then sqlparser would depend on datafusion and not just the other way around.

The dpeendencies on Dialect is maybe harder to imagine

I might be fuzzy on the details as I'm just getting started on the codebase.

BTW to be clear the idea in this PR seems fine to me -- I just figured I would comment

And thank you for commenting!

[alamb]

But then sqlparser would depend on datafusion and not just the other way around.

I meant literally copy/pasting (or some variant thereof) of the relevant structures to break the dependency. Similar to the way @eliaperantoni did in this PR for Span:

• Add related source code locations to errors datafusion#13664