Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CH] CAST to string diff #7602

Open
zhanglistar opened this issue Oct 18, 2024 · 2 comments · May be fixed by #8217
Open

[CH] CAST to string diff #7602

zhanglistar opened this issue Oct 18, 2024 · 2 comments · May be fixed by #8217
Labels
bug Something isn't working triage

Comments

@zhanglistar
Copy link
Contributor

zhanglistar commented Oct 18, 2024 •
edited
Loading

Backend

CH(Clickhouse)

Bug description

SQL:
select cast(array('1','2', 'abc', '\'1\'') as string);

Vanila:
[1, 2, abc, '1']
image

Gluten:
['1','2', 'abc', '\'1\'']
Clickhouse：
image

Spark version

None

Spark configurations

No response

System information

No response

Relevant logs

No response
@zhanglistar zhanglistar added bug Something isn't working triage labels Oct 18, 2024
@zhanglistar
Copy link
Contributor Author

image

@zhanglistar zhanglistar mentioned this issue Oct 23, 2024
Merged
21 tasks
@zhanglistar zhanglistar linked a pull request Dec 12, 2024 that will close this issue
Open
@zhanglistar
Copy link
Contributor Author

select concat(split(n_comment, ' ')) from nation; This sql will throw exception

Caused by: org.apache.gluten.exception.GlutenException: Cannot parse quoted string: expected opening quote ''', got 't': while converting source column `concat(splitByRegexpSpark( _19,n_comment,-1_20))` to destination column `concat(splitByRegexpSpark( _19,n_comment,-1_20))`: while executing 'FUNCTION _CAST(concat(splitByRegexpSpark( _19,n_comment,-1_20)) :: 3, Nullable(Array(String)) :: 4) -> concat(splitByRegexpSpark( _19,n_comment,-1_20)) Nullable(Array(String)) : 2'

Since cast to Array(String)assumes that elements are seperated by quote(that is '), but settings composed_data_type_output_format_mode='spark', discard quote, and error is met.
Modification is not so easy, lots of work need to be done. So we leave it alone now, if we have time, try fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working triage
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[CH] Use spark output format zhanglistar/gluten
1 participant
@zhanglistar