Skip to content

Commit 23e2eee

Browse files
authored
Add RepartitionExec test to projection_pushdown.slt (#20156)
Adding a test that forces a RepartitionExec in the final plan. Our goal will be to get the `get_field(...)` expression pushed through the RepartitionExec into the DataSourceExec
1 parent 2f12f1c commit 23e2eee

File tree

1 file changed

+15
-14
lines changed

1 file changed

+15
-14
lines changed

datafusion/sqllogictest/test_files/projection_pushdown.slt

Lines changed: 15 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1588,22 +1588,23 @@ ORDER BY simple_struct.id;
15881588
4 300 8
15891589
5 250 NULL
15901590

1591-
15921591
#####################
1593-
# Section 13: Cleanup
1592+
# Section 13: RepartitionExec tests
15941593
#####################
15951594

1595+
# Set target partitions to 32 -> this forces a RepartitionExec
15961596
statement ok
1597-
DROP TABLE simple_struct;
1598-
1599-
statement ok
1600-
DROP TABLE nested_struct;
1601-
1602-
statement ok
1603-
DROP TABLE nullable_struct;
1597+
SET datafusion.execution.target_partitions = 32;
16041598

1605-
statement ok
1606-
DROP TABLE multi_struct;
1607-
1608-
statement ok
1609-
DROP TABLE join_right;
1599+
query TT
1600+
EXPLAIN SELECT s['value'] FROM simple_struct WHERE id > 2;
1601+
----
1602+
logical_plan
1603+
01)Projection: get_field(simple_struct.s, Utf8("value"))
1604+
02)--Filter: simple_struct.id > Int64(2)
1605+
03)----TableScan: simple_struct projection=[id, s], partial_filters=[simple_struct.id > Int64(2)]
1606+
physical_plan
1607+
01)ProjectionExec: expr=[get_field(s@0, value) as simple_struct.s[value]]
1608+
02)--FilterExec: id@0 > 2, projection=[s@1]
1609+
03)----RepartitionExec: partitioning=RoundRobinBatch(32), input_partitions=1
1610+
04)------DataSourceExec: file_groups={1 group: [[WORKSPACE_ROOT/datafusion/sqllogictest/test_files/scratch/projection_pushdown/simple.parquet]]}, projection=[id, s], file_type=parquet, predicate=id@0 > 2, pruning_predicate=id_null_count@1 != row_count@2 AND id_max@0 > 2, required_guarantees=[]

0 commit comments

Comments
 (0)