You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Update: reading from table or parquet doesn't cause exponential schema calls growth.
But Queries with literals causes like below does
let sql = "select a, a + 1, a+2 b from (select 1 a union all select 2 a union all select 3 a union all select 4 a union all select 5 a union all select 6 a)";
But this is rare and won't be that critical.
However for reading parquet and tables another problem arises with wild card expansion
adding outer select * adds 50 new calls to with_new_metadata per each *
let sql = "select * from (select * from (select * from (select * from (select * from (select a, a+1, a+2, a+3, a+4, a+5, a+6, a+7, a+8, a+9 from t1)))))";
Is your feature request related to a problem or challenge?
it was found that the planner calling schema transform functions excessively.
Experiment, lets take a simple query
And increase rows in the x subquery by adding more and more rows, and we can see that schema function grows with every record
IMHO That is not expected, once the plan has built the schema calls should not increase with every new record in the dataset
Describe the solution you'd like
Ideally resolve excessive, more real is to reduce such calls
Describe alternatives you've considered
No response
Additional context
Follow up on #9104 investigations
The text was updated successfully, but these errors were encountered: