[SPARK-50582][SQL][PYTHON] Add quote builtin function #49191
Conversation
|
nit: How about adding |
sarutak
changed the title
| | org.apache.spark.sql.catalyst.expressions.Inline | inline | SELECT inline(array(struct(1, 'a'), struct(2, 'b'))) | struct<col1:int,col2:string> | | ||
| | org.apache.spark.sql.catalyst.expressions.Inline | inline_outer | SELECT inline_outer(array(struct(1, 'a'), struct(2, 'b'))) | struct<col1:int,col2:string> | | ||
| | org.apache.spark.sql.catalyst.expressions.InlineExpressionBuilder | inline | SELECT inline(array(struct(1, 'a'), struct(2, 'b'))) | struct<col1:int,col2:string> | | ||
| | org.apache.spark.sql.catalyst.expressions.InlineExpressionBuilder | inline_outer | SELECT inline_outer(array(struct(1, 'a'), struct(2, 'b'))) | struct<col1:int,col2:string> | |
There was a problem hiding this comment.
Is this related to this PR?
There was a problem hiding this comment.
@dongjoon-hyun
Hmm, maybe #48503 should have re-generated the golden file right?
If so, I'll remove this irrelevant change manually, and open another PR for followup.
What do you think?
There was a problem hiding this comment.
Yes, I agree with you. Let's handle them independently.
There was a problem hiding this comment.
Hi, @sarutak . Could you rebase this PR once more since the last test was December.
And, can we remove the irrelevant parts from this file? for example Inline and PodExplode?
- sql/core/src/test/resources/sql-functions/sql-expression-schema.md
| _FUNC_(str) - Returns `str` enclosed by single quotes and | ||
| each instance of single quote in it is preceded by a backslash. |
There was a problem hiding this comment.
It would be better to leave one string because the new line character occurs in docs.
| with UnaryLike[Expression] { | ||
| override def nullIntolerant: Boolean = true | ||
|
|
||
| override lazy val replacement: Expression = Invoke(input, "quote", input.dataType) |
There was a problem hiding this comment.
You mean for instance, implementing quote in ExpressionImplUtils and call it using StaticInvoke right?
| checkAnswer( | ||
| df.selectExpr("quote('Spark')"), | ||
| Row("'Spark'") | ||
| ) | ||
|
|
||
| checkAnswer( | ||
| df.selectExpr("quote(NULL)"), | ||
| Row(null)) |
There was a problem hiding this comment.
This is duplicates of checks in string-functions.sql?
There was a problem hiding this comment.
Yeah, I think string-functions.sql is enough.
How about just removing the duplicates here?
sarutak
force-pushed
the
quote-builtin-function
branch
from
5c7afd9 to
230a967
Compare
What changes were proposed in this pull request?
This PR proposes to add
quotebuiltin function, which encloses the given string by single quotes and prepends a backslash to each instance of single quote in the string.The implementation and behavior are same as Hive's ones.
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=27362046#LanguageManualUDF-StringFunctions
https://github.com/apache/hive/blob/3af4517eb8cfd9407ad34ed78a0b48b57dfaa264/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFQuote.java#L54
Why are the changes needed?
For interoperability with Hive.
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Added new tests.
Was this patch authored or co-authored using generative AI tooling?
No.