<!---
  Licensed to the Apache Software Foundation (ASF) under one
  or more contributor license agreements.  See the NOTICE file
  distributed with this work for additional information
  regarding copyright ownership.  The ASF licenses this file
  to you under the Apache License, Version 2.0 (the
  "License"); you may not use this file except in compliance
  with the License.  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing,
  software distributed under the License is distributed on an
  "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
  KIND, either express or implied.  See the License for the
  specific language governing permissions and limitations
  under the License.
-->

## [16.0.0](https://github.com/apache/datafusion/tree/16.0.0) (2023-01-12)

[Full Changelog](https://github.com/apache/datafusion/compare/16.0.0-rc1...16.0.0)

**Breaking changes:**

- Remove unused ExecutionPlan::relies_input_order \(has been replaced with `required_input_ordering`\) [\#4856](https://github.com/apache/datafusion/pull/4856) ([alamb](https://github.com/alamb))
- Add DataFrame::into_view instead of implementing TableProvider \(\#2659\) [\#4778](https://github.com/apache/datafusion/pull/4778) ([tustvold](https://github.com/tustvold))

**Implemented enhancements:**

- Support custom window frame with AVG aggregate function [\#4845](https://github.com/apache/datafusion/issues/4845)
- add sqllogicaltest for tpch and remove some duplicated test. [\#4801](https://github.com/apache/datafusion/issues/4801)
- Catalog Snapshot Isolation [\#4697](https://github.com/apache/datafusion/issues/4697)
- Support `select .. FROM 'parquet.file'` in datafusion-cli [\#4580](https://github.com/apache/datafusion/issues/4580)

**Fixed bugs:**

- Regression: `write_csv` result has incorrect formatting [\#4876](https://github.com/apache/datafusion/issues/4876)
- Incorrect results for join condition against current master branch [\#4844](https://github.com/apache/datafusion/issues/4844)
- Match Postgres for stddev and variance on less than 3 values [\#4843](https://github.com/apache/datafusion/issues/4843)
- `JOIN ... USING (columns)` works incorrectly with multiple columns \(joined-over columns are missing in the output\) [\#4674](https://github.com/apache/datafusion/issues/4674)
- ROW_NUMBER window function inconsistent across partitions in multi-threaded runtime [\#4673](https://github.com/apache/datafusion/issues/4673)
- `SELECT ... FROM (tbl1 UNION tbl2)` wrongly works like `SELECT DISTINCT ... FROM (tbl1 UNION tbl2)` [\#4667](https://github.com/apache/datafusion/issues/4667)
- DataFrame TableProvider Circular Reference [\#2659](https://github.com/apache/datafusion/issues/2659)

**Documentation updates:**

- Add Synnada to known uses [\#4857](https://github.com/apache/datafusion/pull/4857) ([ozankabak](https://github.com/ozankabak))

**Closed issues:**

- Remove tests from `sql_integration` that were ported to `sqllogictest` [\#4498](https://github.com/apache/datafusion/issues/4498)
- How to register a http url to the `object_store` [\#4491](https://github.com/apache/datafusion/issues/4491)
- optimizer: support `unsigned <-> decimal` for unwrap_cast_in_comparion rule [\#4287](https://github.com/apache/datafusion/issues/4287)
- Add SQL support for NATURAL JOIN [\#117](https://github.com/apache/datafusion/issues/117)
- \[Datafusion\] Datafusion queries involving a column name that begins with a number produces unexpected results [\#108](https://github.com/apache/datafusion/issues/108)

**Merged pull requests:**

- docs: improve `Column::normalize_with_schemas` docs [\#4871](https://github.com/apache/datafusion/pull/4871) ([crepererum](https://github.com/crepererum))
- Skip EliminateCrossJoin rule when meet non-empty join filter [\#4869](https://github.com/apache/datafusion/pull/4869) ([ygf11](https://github.com/ygf11))
- Support for SQL Natural Join [\#4863](https://github.com/apache/datafusion/pull/4863) [[sql](https://github.com/apache/datafusion/labels/sql)] ([Jefffrey](https://github.com/Jefffrey))
- Minor: Move test data into `datafusion/core/tests/data` [\#4855](https://github.com/apache/datafusion/pull/4855) ([alamb](https://github.com/alamb))
- Covariance single row input & null skipping [\#4852](https://github.com/apache/datafusion/pull/4852) ([korowa](https://github.com/korowa))
- Document ability to select directly from files in datafusion-cli [\#4851](https://github.com/apache/datafusion/pull/4851) ([alamb](https://github.com/alamb))
- Fix push_down_projection through a distinct [\#4849](https://github.com/apache/datafusion/pull/4849) ([Jefffrey](https://github.com/Jefffrey))
- Support using var/var_pop/stddev/stddev_pop in window expressions with custom frames [\#4848](https://github.com/apache/datafusion/pull/4848) ([jonmmease](https://github.com/jonmmease))
- Update variance/stddev to work with single values [\#4847](https://github.com/apache/datafusion/pull/4847) ([jonmmease](https://github.com/jonmmease))
- Implement retract_batch for AvgAccumulator [\#4846](https://github.com/apache/datafusion/pull/4846) ([jonmmease](https://github.com/jonmmease))
- Support wildcard select on multiple column using joins [\#4840](https://github.com/apache/datafusion/pull/4840) [[sql](https://github.com/apache/datafusion/labels/sql)] ([Jefffrey](https://github.com/Jefffrey))
- Orthogonalize distribution and sort enforcement rules into `EnforceDistribution` and `EnforceSorting` [\#4839](https://github.com/apache/datafusion/pull/4839) ([mustafasrepo](https://github.com/mustafasrepo))
- support `select .. FROM 'parquet.file'` in datafusion-cli [\#4838](https://github.com/apache/datafusion/pull/4838) ([unconsolable](https://github.com/unconsolable))
- Remove tests from sql_integration that were ported to sqllogictest [\#4836](https://github.com/apache/datafusion/pull/4836) ([matthewwillian](https://github.com/matthewwillian))
- add tpch sqllogicaltest and remove some duplicated test [\#4802](https://github.com/apache/datafusion/pull/4802) ([jackwener](https://github.com/jackwener))

## [16.0.0-rc1](https://github.com/apache/datafusion/tree/16.0.0-rc1) (2023-01-07)

[Full Changelog](https://github.com/apache/datafusion/compare/15.0.0...16.0.0-rc1)

**Breaking changes:**

- Enable PhysicalOptimizerRule lazily \(\#4806\) [\#4807](https://github.com/apache/datafusion/pull/4807) ([tustvold](https://github.com/tustvold))
- Move ConfigOptions to core [\#4803](https://github.com/apache/datafusion/pull/4803) [[sql](https://github.com/apache/datafusion/labels/sql)] ([tustvold](https://github.com/tustvold))
- remove Operator::{Like,NotLike,ILike,NotILike} [\#4792](https://github.com/apache/datafusion/pull/4792) ([unconsolable](https://github.com/unconsolable))
- Move subquery alias assignment onto rules [\#4767](https://github.com/apache/datafusion/pull/4767) ([tustvold](https://github.com/tustvold))
- Make SessionState members private [\#4764](https://github.com/apache/datafusion/pull/4764) ([tustvold](https://github.com/tustvold))
- Deprecate SessionContext physical plan methods \(\#4617\) [\#4751](https://github.com/apache/datafusion/pull/4751) ([tustvold](https://github.com/tustvold))
- Decouple physical optimizer from SessionConfig \(\#3887\) [\#4749](https://github.com/apache/datafusion/pull/4749) ([tustvold](https://github.com/tustvold))
- Don't share ConfigOptions \(\#3886\) [\#4712](https://github.com/apache/datafusion/pull/4712) ([tustvold](https://github.com/tustvold))
- Push SessionState into FileFormat \(\#4349\) [\#4699](https://github.com/apache/datafusion/pull/4699) ([tustvold](https://github.com/tustvold))
- Make SessionContext members private [\#4698](https://github.com/apache/datafusion/pull/4698) ([tustvold](https://github.com/tustvold))
- Make OptimizerConfig a trait \(\#4631\) \(\#4638\) [\#4645](https://github.com/apache/datafusion/pull/4645) ([tustvold](https://github.com/tustvold))
- DataFrame owned SessionState [\#4633](https://github.com/apache/datafusion/pull/4633) ([tustvold](https://github.com/tustvold))
- Make LogicalPlanBuilder consuming \(\#4622\) [\#4632](https://github.com/apache/datafusion/pull/4632) [[sql](https://github.com/apache/datafusion/labels/sql)] ([tustvold](https://github.com/tustvold))
- Make DataFrame API consuming \(\#4621\) [\#4624](https://github.com/apache/datafusion/pull/4624) ([tustvold](https://github.com/tustvold))
- Make execute_stream functions sync [\#4608](https://github.com/apache/datafusion/pull/4608) ([tustvold](https://github.com/tustvold))
- Remove ObjectStore from FileStream \(\#4533\) [\#4601](https://github.com/apache/datafusion/pull/4601) ([tustvold](https://github.com/tustvold))
- Remove `AggregateState` wrapper [\#4582](https://github.com/apache/datafusion/pull/4582) ([alamb](https://github.com/alamb))
- Fix querying and defining table / view names with period [\#4530](https://github.com/apache/datafusion/pull/4530) [[sql](https://github.com/apache/datafusion/labels/sql)] ([alamb](https://github.com/alamb))
- refactor code about `subquery_alias` and `expr-alias`. [\#4451](https://github.com/apache/datafusion/pull/4451) [[sql](https://github.com/apache/datafusion/labels/sql)] ([jackwener](https://github.com/jackwener))

**Implemented enhancements:**

- Move the ExtractEquijoinPredicate behind the SubqueryFilterToJoin [\#4759](https://github.com/apache/datafusion/issues/4759)
- Remove the config `datafusion.execution.coalesce_target_batch_size` [\#4756](https://github.com/apache/datafusion/issues/4756)
- SimplifyExpressions will fail when rebuild equijoin with alias [\#4754](https://github.com/apache/datafusion/issues/4754)
- Provide a constructor for the ConfigOptions with HashMap\<String, String\> [\#4752](https://github.com/apache/datafusion/issues/4752)
- Non-deprecated support for planning SQL without DDL [\#4720](https://github.com/apache/datafusion/issues/4720)
- Add regression tests for planning TPC-DS queries [\#4718](https://github.com/apache/datafusion/issues/4718)
- Move the extracting join keys logic to optimizer [\#4710](https://github.com/apache/datafusion/issues/4710)
- Support compression in `IPCWriter` [\#4708](https://github.com/apache/datafusion/issues/4708)
- Support prepared statement parameter type inference [\#4700](https://github.com/apache/datafusion/issues/4700)
- PruningPredicate Use Physical not Logical Predicate [\#4695](https://github.com/apache/datafusion/issues/4695)
- Support for executing infinite files [\#4692](https://github.com/apache/datafusion/issues/4692)
- Add a sort rule to remove unnecessary SortExecs from physical plan [\#4686](https://github.com/apache/datafusion/issues/4686)
- Install `protoc` automatically when building `datafusion/proto` crate [\#4684](https://github.com/apache/datafusion/issues/4684)
- Make DfSchema wrap SchemaRef [\#4680](https://github.com/apache/datafusion/issues/4680)
- Reorder the physical plan optimizer rules [\#4678](https://github.com/apache/datafusion/issues/4678)
- Inconsistent behavior with PostgreSQL to decide Window Expressions ordering [\#4641](https://github.com/apache/datafusion/issues/4641)
- Returns error too late when parsing invalid file compression type. [\#4636](https://github.com/apache/datafusion/issues/4636)
- Make OptimizerConfig a Trait [\#4631](https://github.com/apache/datafusion/issues/4631)
- Move Optimize onto DataFrame [\#4626](https://github.com/apache/datafusion/issues/4626)
- Make LogicalPlanBuilder Consuming [\#4622](https://github.com/apache/datafusion/issues/4622)
- Make DataFrame Consuming [\#4621](https://github.com/apache/datafusion/issues/4621)
- rules don't need to recursion inside themselves [\#4613](https://github.com/apache/datafusion/issues/4613)
- \[window function\] support min max with self define sliding window. [\#4603](https://github.com/apache/datafusion/issues/4603)
- Add `try_optimize` for all_rules [\#4598](https://github.com/apache/datafusion/issues/4598)
- Refine the physical plan serialization and deserialization [\#4597](https://github.com/apache/datafusion/issues/4597)
- Normalize datafusion configuration names [\#4595](https://github.com/apache/datafusion/issues/4595)
- Add need_data_exchange in the ExecutionPlan to indicate whether a physical operator needs data exchange [\#4585](https://github.com/apache/datafusion/issues/4585)
- Bump Datafusion sql-parser dependency to 0.28 [\#4573](https://github.com/apache/datafusion/issues/4573)
- tpch test exist duplicated [\#4563](https://github.com/apache/datafusion/issues/4563)
- user-defined aggregate function as window function [\#4552](https://github.com/apache/datafusion/issues/4552)
- Convert a Prepare Logical Plan into a Logical Plan with all parameters replaced with values [\#4550](https://github.com/apache/datafusion/issues/4550)
- FileStream requires fake ObjectStore when ParquetFileReaderFactory is used [\#4533](https://github.com/apache/datafusion/issues/4533)
- Avoid reading the entire file in ChunkedStore [\#4524](https://github.com/apache/datafusion/issues/4524)
- Enrich filter statistics predictions with estimated column boundaries [\#4518](https://github.com/apache/datafusion/issues/4518)
- Show window frame info in physical plan [\#4509](https://github.com/apache/datafusion/issues/4509)
- Add sqllogictest auto labeler [\#4507](https://github.com/apache/datafusion/issues/4507)
- Optimize `is_distinct_from` / `is_not_distinct_from` [\#4482](https://github.com/apache/datafusion/issues/4482)
- Add window func related logic plan to proto ability. [\#4480](https://github.com/apache/datafusion/issues/4480)
- Make window function related struct public. [\#4479](https://github.com/apache/datafusion/issues/4479)
- Improve partition file explain plan display to show groupings [\#4466](https://github.com/apache/datafusion/issues/4466)
- Add support for non-column key for equijoin when eliminating cross join to inner join [\#4442](https://github.com/apache/datafusion/issues/4442)
- Remove the schema checking from `CrossJoinExec::try_new` [\#4431](https://github.com/apache/datafusion/issues/4431)
- Initial support for prepared statement [\#4426](https://github.com/apache/datafusion/issues/4426)
- Add support for NTILE built-in Window Function [\#4403](https://github.com/apache/datafusion/issues/4403)
- Add Support for MIN, MAX Aggregate Functions when run with custom window frames [\#4402](https://github.com/apache/datafusion/issues/4402)
- Support `INSERT INTO` statement [\#4397](https://github.com/apache/datafusion/issues/4397)
- Enhancement: split the SQL `planner` into smaller modules [\#4392](https://github.com/apache/datafusion/issues/4392)
- Proposal: Improve the join keys of logical plan [\#4389](https://github.com/apache/datafusion/issues/4389)
- Add `MergeSubqueryAlias` rule [\#4383](https://github.com/apache/datafusion/issues/4383)
- Optimizer rule support `subqueryAlias` [\#4381](https://github.com/apache/datafusion/issues/4381)
- Rewrite simple regex expressions [\#4370](https://github.com/apache/datafusion/issues/4370)
- Revisit get_statistics_with_limit\(\) method in datasource mod [\#4323](https://github.com/apache/datafusion/issues/4323)
- Support for type coercion for a \(Timestamp, Utf8\) pair [\#4311](https://github.com/apache/datafusion/issues/4311)
- replace the operation about decimal to the arrow-rs kernel [\#4289](https://github.com/apache/datafusion/issues/4289)
- change` date_part` return types to f64 [\#3997](https://github.com/apache/datafusion/issues/3997)
- Better api for setting `ConfigOptions` from SessionContext [\#3908](https://github.com/apache/datafusion/issues/3908)
- Make `ConfigOptions` easier to work with [\#3886](https://github.com/apache/datafusion/issues/3886)
- An asynchronous version of `CatalogList`/`CatalogProvider`/`SchemaProvider` [\#3777](https://github.com/apache/datafusion/issues/3777)
- Allow configs to be set with string values [\#3500](https://github.com/apache/datafusion/issues/3500)
- support scientific notation for SQL literals [\#3448](https://github.com/apache/datafusion/issues/3448)
- Adopt physical plan serde from arrow-ballista [\#3257](https://github.com/apache/datafusion/issues/3257)
- Improve codebase readability and error messages by and consistently handle downcasting [\#3152](https://github.com/apache/datafusion/issues/3152)
- Re-enable where_clauses_object_safety [\#3081](https://github.com/apache/datafusion/issues/3081)
- optimize/simplify the literal data type and remove unnecessary cast、try_cast [\#3031](https://github.com/apache/datafusion/issues/3031)
- Move `datafusion-substrait` crate into `arrow-datafusion` repo [\#2646](https://github.com/apache/datafusion/issues/2646)
- \[enhancement\] rules don't need to recursion inside themselves [\#2620](https://github.com/apache/datafusion/issues/2620)
- Add support for `GROUPING SETS` syntax in SQL planner [\#2469](https://github.com/apache/datafusion/issues/2469)
- Optimize EXISTS subquery expressions by rewriting as semi-join [\#2351](https://github.com/apache/datafusion/issues/2351)
- Add Delta Lake TableProvider [\#525](https://github.com/apache/datafusion/issues/525)
- Support window functions with window frame [\#361](https://github.com/apache/datafusion/issues/361)

**Fixed bugs:**

- PushdownFilter rule exist bug will cause filter change wrong [\#4822](https://github.com/apache/datafusion/issues/4822)
- Unlimited memory consumption in `RepartitionExec` [\#4816](https://github.com/apache/datafusion/issues/4816)
- Physical Optimizer Config Mutation Doesn't Take Effect [\#4806](https://github.com/apache/datafusion/issues/4806)
- cargo test failed `error: linking with `cc` failed: exit status: 1` [\#4790](https://github.com/apache/datafusion/issues/4790)
- Parquet files generated by DataFusion cannot be read by Apache Spark [\#4782](https://github.com/apache/datafusion/issues/4782)
- datafusion-physical-expr doesn't compile when blake3/traits-preview is enabled [\#4781](https://github.com/apache/datafusion/issues/4781)
- Multiple ways to express `like` / `ilike` / `not like` / `not ilike` [\#4765](https://github.com/apache/datafusion/issues/4765)
- SessionState::optimize and SessionState::create_physical_plan Don't Update Query Start Time [\#4747](https://github.com/apache/datafusion/issues/4747)
- Page Filtering Incorrectly Handles Pages with Different Row Counts [\#4744](https://github.com/apache/datafusion/issues/4744)
- cargo test failing on master due to tpcds_logical_q41 stackoverflow [\#4728](https://github.com/apache/datafusion/issues/4728)
- PruningPredicate Different Evaluation Context from Query [\#4693](https://github.com/apache/datafusion/issues/4693)
- Skipping optimizer rule due to create_name not supporting wildcard [\#4681](https://github.com/apache/datafusion/issues/4681)
- Create physical plan bug: got Arrow schema with 1 and DataFusion schema with 0 [\#4677](https://github.com/apache/datafusion/issues/4677)
- Timestamp \<-\> Date32 compare doesn't work [\#4672](https://github.com/apache/datafusion/issues/4672)
- Wrongly use the function `clamp` [\#4654](https://github.com/apache/datafusion/issues/4654)
- Fix the clippy errors [\#4653](https://github.com/apache/datafusion/issues/4653)
- Filter Null Keys Update Not Taking Effect [\#4638](https://github.com/apache/datafusion/issues/4638)
- Should not generate duplicate sort keys from Window expr's partition by keys [\#4635](https://github.com/apache/datafusion/issues/4635)
- `common_sub_expression_eliminate` exists bug [\#4575](https://github.com/apache/datafusion/issues/4575)
- Confusing "Bare" in doesn't exist messages [\#4571](https://github.com/apache/datafusion/issues/4571)
- `having` shouldn't include alias in projection [\#4556](https://github.com/apache/datafusion/issues/4556)
- wrong comment about having [\#4554](https://github.com/apache/datafusion/issues/4554)
- `drop view t1, t2, ...` and `drop table t1, t2, ...` silently ignores arguments past the first [\#4531](https://github.com/apache/datafusion/issues/4531)
- Extract from timestamp doesn't support nanosecond [\#4528](https://github.com/apache/datafusion/issues/4528)
- `prepare_select_exprs` don't need `outer_query_schema` [\#4526](https://github.com/apache/datafusion/issues/4526)
- Table names with periods are not handled correctly [\#4513](https://github.com/apache/datafusion/issues/4513)
- `Push_down_projection` push redundant column. [\#4486](https://github.com/apache/datafusion/issues/4486)
- Planner don't generate `SubqueryAlias` [\#4483](https://github.com/apache/datafusion/issues/4483)
- Planner generate replicated `Projection` | `SubqueryAlias` [\#4481](https://github.com/apache/datafusion/issues/4481)
- `apply_table_alias` will ignore alias_name when columns is empty. [\#4454](https://github.com/apache/datafusion/issues/4454)
- Fix output_ordering of WindowAggExec [\#4438](https://github.com/apache/datafusion/issues/4438)
- Incorrect error for plus/minus operations over timestamps and dates [\#4420](https://github.com/apache/datafusion/issues/4420)
- Optimization rule `filter_push_down` causes `FieldNotFound` error [\#4401](https://github.com/apache/datafusion/issues/4401)
- Should not convert a normal non-inner join to Cross Join when there are non-equal Join conditions [\#4363](https://github.com/apache/datafusion/issues/4363)
- MemoryConsumer::try_grow Underflow [\#4328](https://github.com/apache/datafusion/issues/4328)
- Potential MemoryManager Deadlock [\#4325](https://github.com/apache/datafusion/issues/4325)
- `create external table` should fail to parse if syntax is incorrect [\#4262](https://github.com/apache/datafusion/issues/4262)
- Nullif func states support for Boolean type, but fails if this is attempted [\#4205](https://github.com/apache/datafusion/issues/4205)
- `ProjectionPushDown` rule don't consider the alias in projection. [\#4174](https://github.com/apache/datafusion/issues/4174)
- Stack overflow planning complex query [\#4065](https://github.com/apache/datafusion/issues/4065)
- Can not use `extract <part>` on the value of `now()` [\#3980](https://github.com/apache/datafusion/issues/3980)
- Bug with intervals and logical and/or [\#3944](https://github.com/apache/datafusion/issues/3944)
- CoalesceBatches doesn't provide correct elapsed_compute info in metrics [\#3894](https://github.com/apache/datafusion/issues/3894)
- Paniced at to_timestamp_micros function when the timestamp is too large. [\#3832](https://github.com/apache/datafusion/issues/3832)
- Optimizer casts decimals to different values on different platforms [\#3791](https://github.com/apache/datafusion/issues/3791)
- CSV inference reads in the whole file to memory, regardless of row limit [\#3658](https://github.com/apache/datafusion/issues/3658)
- after type coercion `CommonSubexprEliminate` will produce invalid projection [\#3635](https://github.com/apache/datafusion/issues/3635)
- panic at `attempt to multiply with overflow` when doing math on Decimal128 columns [\#3437](https://github.com/apache/datafusion/issues/3437)
- Precedence bug with date comparison to date plus interval [\#3408](https://github.com/apache/datafusion/issues/3408)
- Median aggregation using DataFrame panics: "AggregateState is not a scalar aggregate" [\#3105](https://github.com/apache/datafusion/issues/3105)
- `date_part` does't work for `now()` [\#3096](https://github.com/apache/datafusion/issues/3096)
- hash_join panics when join keys have different data types [\#2877](https://github.com/apache/datafusion/issues/2877)
- Memory manager triggers unnecessary spills [\#2829](https://github.com/apache/datafusion/issues/2829)
- Address performance/execution plan of TPCH query 9 [\#77](https://github.com/apache/datafusion/issues/77)

**Documentation updates:**

- Add a new open source project that is use DataFusion as query engine [\#4768](https://github.com/apache/datafusion/pull/4768) ([francis-du](https://github.com/francis-du))

**Closed issues:**

- move the tests in planner [\#4798](https://github.com/apache/datafusion/issues/4798)
- Make it easier to update sqltestlogic test expected output \("test script completion mode"\) [\#4570](https://github.com/apache/datafusion/issues/4570)
- Make ConfigOption names into an Enum [\#4517](https://github.com/apache/datafusion/issues/4517)
- Implement null / empty string handling for sqllogictest [\#4500](https://github.com/apache/datafusion/issues/4500)
- Write a blog about parquet predicate pushdown [\#3464](https://github.com/apache/datafusion/issues/3464)
- Ensure column names are equivalent with or without optimization [\#1123](https://github.com/apache/datafusion/issues/1123)

**Merged pull requests:**

- Bump tokio from 1.23.0 to 1.23.1 in /datafusion-cli [\#4835](https://github.com/apache/datafusion/pull/4835) ([dependabot[bot]](https://github.com/apps/dependabot))
- Fix a few links in `roadmap.md` [\#4833](https://github.com/apache/datafusion/pull/4833) ([romanz](https://github.com/romanz))
- DataFusion 16.0.0 release prep: Update version + add changelog [\#4831](https://github.com/apache/datafusion/pull/4831) [[sql](https://github.com/apache/datafusion/labels/sql)] ([andygrove](https://github.com/andygrove))
- feat: use arrow row format for hash-group-by [\#4830](https://github.com/apache/datafusion/pull/4830) ([crepererum](https://github.com/crepererum))
- refactor: split relation of planner into one part. [\#4829](https://github.com/apache/datafusion/pull/4829) [[sql](https://github.com/apache/datafusion/labels/sql)] ([jackwener](https://github.com/jackwener))
- bugfix: remove cnf_rewrite in push_down_filter [\#4825](https://github.com/apache/datafusion/pull/4825) ([jackwener](https://github.com/jackwener))
- minor: add some comments to row group pruning tests [\#4823](https://github.com/apache/datafusion/pull/4823) ([alamb](https://github.com/alamb))
- Handle trailing tbl column in TPCH benchmarks [\#4821](https://github.com/apache/datafusion/pull/4821) ([tustvold](https://github.com/tustvold))
- fix: account for memory in `RepartitionExec` [\#4820](https://github.com/apache/datafusion/pull/4820) ([crepererum](https://github.com/crepererum))
- Fix clippy [\#4817](https://github.com/apache/datafusion/pull/4817) ([tustvold](https://github.com/tustvold))
- Add test cases: row group filter with missing statistics for decimal data type [\#4810](https://github.com/apache/datafusion/pull/4810) ([liukun4515](https://github.com/liukun4515))
- Move default catalog and schema onto ConfigOptions \(\#3887\) [\#4805](https://github.com/apache/datafusion/pull/4805) ([tustvold](https://github.com/tustvold))
- remove duplicated test [\#4800](https://github.com/apache/datafusion/pull/4800) ([jackwener](https://github.com/jackwener))
- Update sqlparser requirement from 0.29 to 0.30 [\#4799](https://github.com/apache/datafusion/pull/4799) [[sql](https://github.com/apache/datafusion/labels/sql)] ([dependabot[bot]](https://github.com/apps/dependabot))
- rewrite the function `ensure_any_column_reference_is_unambiguous` [\#4797](https://github.com/apache/datafusion/pull/4797) [[sql](https://github.com/apache/datafusion/labels/sql)] ([HaoYang670](https://github.com/HaoYang670))
- Uncomment nanoseconds tests after sql parser upgrade [\#4789](https://github.com/apache/datafusion/pull/4789) ([comphead](https://github.com/comphead))
- fix: ListingSchemaProvider directory paths \(related: \#4204\) [\#4788](https://github.com/apache/datafusion/pull/4788) ([cfraz89](https://github.com/cfraz89))
- Minimize stack space required to plan deeply nested binary expressions [\#4787](https://github.com/apache/datafusion/pull/4787) [[sql](https://github.com/apache/datafusion/labels/sql)] ([alamb](https://github.com/alamb))
- Minor: Refactor some sql planning code into functions [\#4785](https://github.com/apache/datafusion/pull/4785) [[sql](https://github.com/apache/datafusion/labels/sql)] ([alamb](https://github.com/alamb))
- Make datafusion-physical-expr compatible with blake3/traits-preview feature. [\#4784](https://github.com/apache/datafusion/pull/4784) ([BoredPerson](https://github.com/BoredPerson))
- refactor: split expression pf planner into one part. [\#4783](https://github.com/apache/datafusion/pull/4783) [[sql](https://github.com/apache/datafusion/labels/sql)] ([jackwener](https://github.com/jackwener))
- Fix Stack overflow in sql planning in debug builds [\#4779](https://github.com/apache/datafusion/pull/4779) [[sql](https://github.com/apache/datafusion/labels/sql)] ([alamb](https://github.com/alamb))
- Pipeline-friendly Bounded Memory Window Executor [\#4777](https://github.com/apache/datafusion/pull/4777) ([mustafasrepo](https://github.com/mustafasrepo))
- Implement OptimizerConfig for SessionState [\#4775](https://github.com/apache/datafusion/pull/4775) ([tustvold](https://github.com/tustvold))
- refactor: extract `parse_value` [\#4774](https://github.com/apache/datafusion/pull/4774) [[sql](https://github.com/apache/datafusion/labels/sql)] ([jackwener](https://github.com/jackwener))
- Structify ConfigOptions \(\#4517\) [\#4771](https://github.com/apache/datafusion/pull/4771) ([tustvold](https://github.com/tustvold))
- Update sqlparser to `29.0.0` [\#4770](https://github.com/apache/datafusion/pull/4770) [[sql](https://github.com/apache/datafusion/labels/sql)] ([alamb](https://github.com/alamb))
- Refactor extract_join_keys and move the ExtractEquijoinPredicate rule [\#4760](https://github.com/apache/datafusion/pull/4760) ([ygf11](https://github.com/ygf11))
- Remove the config datafusion.execution.coalesce_target_batch_size and use datafusion.execution.batch_size instead [\#4757](https://github.com/apache/datafusion/pull/4757) ([yahoNanJing](https://github.com/yahoNanJing))
- Add alias check for equijoin in from_plan [\#4755](https://github.com/apache/datafusion/pull/4755) ([ygf11](https://github.com/ygf11))
- Take the top level `schema` into account when creating `UnionExec` [\#4753](https://github.com/apache/datafusion/pull/4753) ([HaoYang670](https://github.com/HaoYang670))
- Set query_execution_start_time on snapshot from SessionContext \(\#4747\) [\#4750](https://github.com/apache/datafusion/pull/4750) ([tustvold](https://github.com/tustvold))
- minor: Improve docstrings [\#4748](https://github.com/apache/datafusion/pull/4748) [[sql](https://github.com/apache/datafusion/labels/sql)] ([alamb](https://github.com/alamb))
- Append generated column to the schema instead of prepending for WindowAggExec [\#4746](https://github.com/apache/datafusion/pull/4746) ([mustafasrepo](https://github.com/mustafasrepo))
- Minor: comments about coercion in physical planner [\#4745](https://github.com/apache/datafusion/pull/4745) ([alamb](https://github.com/alamb))
- Simplify parquet filter predicate test, fix Page Filtering Incorrectly Handles Pages with Different Row Counts [\#4743](https://github.com/apache/datafusion/pull/4743) ([tustvold](https://github.com/tustvold))
- support byte array for decimal in parquet page and row group filters [\#4742](https://github.com/apache/datafusion/pull/4742) ([liukun4515](https://github.com/liukun4515))
- revert some code for \#4726 / remove unnecessary coercion in physical plans [\#4741](https://github.com/apache/datafusion/pull/4741) ([liukun4515](https://github.com/liukun4515))
- Cleanup InformationSchema plumbing [\#4740](https://github.com/apache/datafusion/pull/4740) ([tustvold](https://github.com/tustvold))
- Minor: use a common method to check the validate of equijoin predicate [\#4739](https://github.com/apache/datafusion/pull/4739) ([ygf11](https://github.com/ygf11))
- minor: Support more data type for `null_counts` in the `PruningStatistics` [\#4738](https://github.com/apache/datafusion/pull/4738) ([liukun4515](https://github.com/liukun4515))
- Extended datatypes & signatures support for `NULLIF` function [\#4737](https://github.com/apache/datafusion/pull/4737) ([korowa](https://github.com/korowa))
- minor: improve debug logging for pruning predicates [\#4736](https://github.com/apache/datafusion/pull/4736) ([alamb](https://github.com/alamb))
- refactor: parallelize `parquet_exec` test case `single_file` [\#4735](https://github.com/apache/datafusion/pull/4735) ([waynexia](https://github.com/waynexia))
- fix: add one more projection to recover output schema [\#4733](https://github.com/apache/datafusion/pull/4733) ([waynexia](https://github.com/waynexia))
- remove `SubqueryFilterToJoin` [\#4731](https://github.com/apache/datafusion/pull/4731) ([jackwener](https://github.com/jackwener))
- Create writer with `arrow::ipc::IPCWriteOptions` [\#4730](https://github.com/apache/datafusion/pull/4730) ([askoa](https://github.com/askoa))
- Implement cast between Date and Timestamp [\#4726](https://github.com/apache/datafusion/pull/4726) ([comphead](https://github.com/comphead))
- Dynamic information_schema configuration and port more tests [\#4722](https://github.com/apache/datafusion/pull/4722) ([alamb](https://github.com/alamb))
- Add TPC-DS query planning regression tests [\#4719](https://github.com/apache/datafusion/pull/4719) ([andygrove](https://github.com/andygrove))
- Minor: refactor streaming CSV inference code [\#4717](https://github.com/apache/datafusion/pull/4717) ([alamb](https://github.com/alamb))
- Reorder the physical plan optimizer rules, extract `GlobalSortSelection`, make `Repartition` optional [\#4714](https://github.com/apache/datafusion/pull/4714) ([yahoNanJing](https://github.com/yahoNanJing))
- Eagerly construct PagePruningPredicate [\#4713](https://github.com/apache/datafusion/pull/4713) ([tustvold](https://github.com/tustvold))
- Move the extract_join_keys to optimizer [\#4711](https://github.com/apache/datafusion/pull/4711) [[sql](https://github.com/apache/datafusion/labels/sql)] ([ygf11](https://github.com/ygf11))
- Avoid to bypass `try_new/new()` to build plan directly and cleanup filter [\#4702](https://github.com/apache/datafusion/pull/4702) ([jackwener](https://github.com/jackwener))
- MINOR: Remove where_clause_object_safety clippy ignore \(\#3081\) [\#4696](https://github.com/apache/datafusion/pull/4696) ([tustvold](https://github.com/tustvold))
- Support for executing infinite files and boundedness-aware join reordering rule [\#4694](https://github.com/apache/datafusion/pull/4694) ([metesynnada](https://github.com/metesynnada))
- Unnecessary SortExec removal rule from Physical Plan [\#4691](https://github.com/apache/datafusion/pull/4691) ([mustafasrepo](https://github.com/mustafasrepo))
- minor: rename the github actions [\#4689](https://github.com/apache/datafusion/pull/4689) ([jackwener](https://github.com/jackwener))
- FOLLOWUP: remove more recursion in optimizer rules. [\#4687](https://github.com/apache/datafusion/pull/4687) ([jackwener](https://github.com/jackwener))
- Add line that prevents display_name from being called on Wildcard [\#4682](https://github.com/apache/datafusion/pull/4682) ([andre-cc-natzka](https://github.com/andre-cc-natzka))
- Deprecate SessionContext::create_logical_plan \(\#4617\) [\#4679](https://github.com/apache/datafusion/pull/4679) ([tustvold](https://github.com/tustvold))
- Support `NTILE` window function [\#4676](https://github.com/apache/datafusion/pull/4676) ([berkaycpp](https://github.com/berkaycpp))
- Support min max aggregates in window functions with sliding windows [\#4675](https://github.com/apache/datafusion/pull/4675) ([berkaycpp](https://github.com/berkaycpp))
- Refactor Expr::AggregateFunction and Expr::WindowFunction to use struct [\#4671](https://github.com/apache/datafusion/pull/4671) [[sql](https://github.com/apache/datafusion/labels/sql)] ([Jefffrey](https://github.com/Jefffrey))
- Support type coercion for equijoin [\#4666](https://github.com/apache/datafusion/pull/4666) ([ygf11](https://github.com/ygf11))
- Add `--complete` auto completion mode to `sqllogictests` [\#4665](https://github.com/apache/datafusion/pull/4665) ([alamb](https://github.com/alamb))
- Fix CoalesceBatches elasped_compute metric [\#4664](https://github.com/apache/datafusion/pull/4664) ([Jefffrey](https://github.com/Jefffrey))
- Refactor Expr::Sort to use struct [\#4663](https://github.com/apache/datafusion/pull/4663) [[sql](https://github.com/apache/datafusion/labels/sql)] ([Jefffrey](https://github.com/Jefffrey))
- More descriptive error for plus/minus between timestamps/dates [\#4662](https://github.com/apache/datafusion/pull/4662) ([Jefffrey](https://github.com/Jefffrey))
- Stream CSV file during schema inference [\#4661](https://github.com/apache/datafusion/pull/4661) ([Jefffrey](https://github.com/Jefffrey))
- Refine the logical and physical plan serialization and deserialization [\#4659](https://github.com/apache/datafusion/pull/4659) ([yahoNanJing](https://github.com/yahoNanJing))
- Use thiserror in sqllogictest erorr [\#4657](https://github.com/apache/datafusion/pull/4657) ([xudong963](https://github.com/xudong963))
- fix `cargo clippy` warning [\#4652](https://github.com/apache/datafusion/pull/4652) [[sql](https://github.com/apache/datafusion/labels/sql)] ([jackwener](https://github.com/jackwener))
- Improve group by hash performance: avoid group-key/-state clones for hash-groupby [\#4651](https://github.com/apache/datafusion/pull/4651) ([crepererum](https://github.com/crepererum))
- remove recursion in optimizer rules [\#4650](https://github.com/apache/datafusion/pull/4650) ([jackwener](https://github.com/jackwener))
- replace the arithmetic op for decimal array op decimal array using arrow kernel [\#4648](https://github.com/apache/datafusion/pull/4648) ([liukun4515](https://github.com/liukun4515))
- simplify regex expressions [\#4646](https://github.com/apache/datafusion/pull/4646) ([crepererum](https://github.com/crepererum))
- Avoid generate duplicate sort Keys from Window Expressions, fix bug when decide Window Expressions ordering [\#4643](https://github.com/apache/datafusion/pull/4643) [[sql](https://github.com/apache/datafusion/labels/sql)] ([mingmwang](https://github.com/mingmwang))
- Refactor Expr::TryCast to use a struct [\#4642](https://github.com/apache/datafusion/pull/4642) [[sql](https://github.com/apache/datafusion/labels/sql)] ([ygf11](https://github.com/ygf11))
- add `ILIKE` support [\#4639](https://github.com/apache/datafusion/pull/4639) ([crepererum](https://github.com/crepererum))
- Detect invalid \(unsupported\) compression types when parsing [\#4637](https://github.com/apache/datafusion/pull/4637) [[sql](https://github.com/apache/datafusion/labels/sql)] ([HaoYang670](https://github.com/HaoYang670))
- unwrap_cast_in_comparison.rs: support unint \<-\> decimal [\#4634](https://github.com/apache/datafusion/pull/4634) ([liukun4515](https://github.com/liukun4515))
- MINOR: Fix incorrect config definitions [\#4623](https://github.com/apache/datafusion/pull/4623) ([andygrove](https://github.com/andygrove))
- FOLLOWUP: remove `optimize()` [\#4619](https://github.com/apache/datafusion/pull/4619) ([jackwener](https://github.com/jackwener))
- Optimizer: avoid every rule must recursive children in optimizer [\#4618](https://github.com/apache/datafusion/pull/4618) ([jackwener](https://github.com/jackwener))
- fix: run logical optimizer rules for `TableScan` expressions [\#4614](https://github.com/apache/datafusion/pull/4614) ([crepererum](https://github.com/crepererum))
- refactor: relax the signature of register\_\* in SessionContext [\#4612](https://github.com/apache/datafusion/pull/4612) ([waynexia](https://github.com/waynexia))
- Remove the function `consume_token` from the parser [\#4609](https://github.com/apache/datafusion/pull/4609) [[sql](https://github.com/apache/datafusion/labels/sql)] ([HaoYang670](https://github.com/HaoYang670))
- Make SchemaProvider::table async [\#4607](https://github.com/apache/datafusion/pull/4607) ([tustvold](https://github.com/tustvold))
- Lazy system tables [\#4606](https://github.com/apache/datafusion/pull/4606) ([tustvold](https://github.com/tustvold))
- Refactor: Change equijoin keys from column to expression in logical join [\#4602](https://github.com/apache/datafusion/pull/4602) [[sql](https://github.com/apache/datafusion/labels/sql)] ([ygf11](https://github.com/ygf11))
- refactor: extract `assert_optimized_plan_eq` from UT. [\#4600](https://github.com/apache/datafusion/pull/4600) ([jackwener](https://github.com/jackwener))
- add `try_optimize()` for all rules. [\#4599](https://github.com/apache/datafusion/pull/4599) ([jackwener](https://github.com/jackwener))
- Normalize datafusion configuration names [\#4596](https://github.com/apache/datafusion/pull/4596) ([yahoNanJing](https://github.com/yahoNanJing))
- Fix the bugs in parsing `COMPRESSION TYPE` [\#4590](https://github.com/apache/datafusion/pull/4590) [[sql](https://github.com/apache/datafusion/labels/sql)] ([HaoYang670](https://github.com/HaoYang670))
- Minor: Remove datafusion-core dev dependency from datafusion-sql [\#4589](https://github.com/apache/datafusion/pull/4589) [[sql](https://github.com/apache/datafusion/labels/sql)] ([alamb](https://github.com/alamb))
- Improve error handling for array downcasting [\#4588](https://github.com/apache/datafusion/pull/4588) ([retikulum](https://github.com/retikulum))
- Update to arrow v29 [\#4587](https://github.com/apache/datafusion/pull/4587) [[sql](https://github.com/apache/datafusion/labels/sql)] ([tustvold](https://github.com/tustvold))
- Add need_data_exchange in the ExecutionPlan to indicate whether a physical operator needs data exchange [\#4586](https://github.com/apache/datafusion/pull/4586) ([yahoNanJing](https://github.com/yahoNanJing))
- Move subset of select tests to sqllogic [\#4583](https://github.com/apache/datafusion/pull/4583) ([ajayaa](https://github.com/ajayaa))
- bugfix: just allow having use expr in `groupby` or `aggr` [\#4579](https://github.com/apache/datafusion/pull/4579) [[sql](https://github.com/apache/datafusion/labels/sql)] ([jackwener](https://github.com/jackwener))
- Output sqllogictests with arrow display rather than CSV writer [\#4578](https://github.com/apache/datafusion/pull/4578) ([alamb](https://github.com/alamb))
- Minor: Add test case for reduce cross join [\#4577](https://github.com/apache/datafusion/pull/4577) ([ygf11](https://github.com/ygf11))
- refactor: remove redundant `outer_query_schema` [\#4576](https://github.com/apache/datafusion/pull/4576) [[sql](https://github.com/apache/datafusion/labels/sql)] ([jackwener](https://github.com/jackwener))
- Preserve the TryCast expression in columnize_expr [\#4574](https://github.com/apache/datafusion/pull/4574) [[sql](https://github.com/apache/datafusion/labels/sql)] ([byteink](https://github.com/byteink))
- Remove Confusing "Bare" in does not exist messages [\#4572](https://github.com/apache/datafusion/pull/4572) [[sql](https://github.com/apache/datafusion/labels/sql)] ([alamb](https://github.com/alamb))
- Minor: Add tests for date interval predicate handling [\#4569](https://github.com/apache/datafusion/pull/4569) ([alamb](https://github.com/alamb))
- Update sqlparser requirement from 0.27 to 0.28 [\#4568](https://github.com/apache/datafusion/pull/4568) [[sql](https://github.com/apache/datafusion/labels/sql)] ([alamb](https://github.com/alamb))
- Avoid materializing local varaibles when creating `sortMergeJoinExec` [\#4566](https://github.com/apache/datafusion/pull/4566) ([HaoYang670](https://github.com/HaoYang670))
- Minor: Fix logical conflict [\#4565](https://github.com/apache/datafusion/pull/4565) ([alamb](https://github.com/alamb))
- feat: support nested loop join with the initial version [\#4562](https://github.com/apache/datafusion/pull/4562) [[sql](https://github.com/apache/datafusion/labels/sql)] ([liukun4515](https://github.com/liukun4515))
- feat: prepare logical plan to logical plan without params/placeholders [\#4561](https://github.com/apache/datafusion/pull/4561) [[sql](https://github.com/apache/datafusion/labels/sql)] ([NGA-TRAN](https://github.com/NGA-TRAN))
- Write faster kernel for is_distinct [\#4560](https://github.com/apache/datafusion/pull/4560) ([comphead](https://github.com/comphead))
- refactor code about `query -> plan` for subqueries [\#4559](https://github.com/apache/datafusion/pull/4559) [[sql](https://github.com/apache/datafusion/labels/sql)] ([jackwener](https://github.com/jackwener))
- fix: remove wrong comment about `having` [\#4555](https://github.com/apache/datafusion/pull/4555) [[sql](https://github.com/apache/datafusion/labels/sql)] ([jackwener](https://github.com/jackwener))
- feat: user-defined aggregate function\(UDAF\) as window function [\#4553](https://github.com/apache/datafusion/pull/4553) [[sql](https://github.com/apache/datafusion/labels/sql)] ([MichaelScofield](https://github.com/MichaelScofield))
- Fix date_part/extract functions to support now\(\) [\#4548](https://github.com/apache/datafusion/pull/4548) ([comphead](https://github.com/comphead))
- bump sqllogictest to 0.9.0 [\#4547](https://github.com/apache/datafusion/pull/4547) ([xxchan](https://github.com/xxchan))
- minor: Remove more clones from the planner [\#4546](https://github.com/apache/datafusion/pull/4546) [[sql](https://github.com/apache/datafusion/labels/sql)] ([alamb](https://github.com/alamb))
- Add tests for coercion of timestamps to strings [\#4545](https://github.com/apache/datafusion/pull/4545) ([alamb](https://github.com/alamb))
- MINOR: move sqllogictest to dev-dependencies [\#4544](https://github.com/apache/datafusion/pull/4544) ([alamb](https://github.com/alamb))
- MINOR: add some comments about intended use of ChunkedStore [\#4541](https://github.com/apache/datafusion/pull/4541) ([alamb](https://github.com/alamb))
- fix: remove TODOs linked to arrow\#3147 [\#4540](https://github.com/apache/datafusion/pull/4540) ([crepererum](https://github.com/crepererum))
- refactor: remove redundant `build_join_schema()` [\#4538](https://github.com/apache/datafusion/pull/4538) ([jackwener](https://github.com/jackwener))
- Move some create/drop tests to `ddl.slt` [\#4535](https://github.com/apache/datafusion/pull/4535) ([alamb](https://github.com/alamb))
- Minor: Avoid cloning as many `Ident` during SQL planning [\#4534](https://github.com/apache/datafusion/pull/4534) [[sql](https://github.com/apache/datafusion/labels/sql)] ([alamb](https://github.com/alamb))
- shouldn't add `outer_query_schema` in `sql_select_to_rex` [\#4527](https://github.com/apache/datafusion/pull/4527) [[sql](https://github.com/apache/datafusion/labels/sql)] ([jackwener](https://github.com/jackwener))
- Avoid reading the entire file in ChunkedStore [\#4525](https://github.com/apache/datafusion/pull/4525) ([metesynnada](https://github.com/metesynnada))
- Simplify MemoryManager [\#4522](https://github.com/apache/datafusion/pull/4522) ([tustvold](https://github.com/tustvold))
- Fix limited statistic collection accross files with no stats [\#4521](https://github.com/apache/datafusion/pull/4521) ([isidentical](https://github.com/isidentical))
- refactor: make Ctes a struct to also store data types provided by prepare stmt [\#4520](https://github.com/apache/datafusion/pull/4520) [[sql](https://github.com/apache/datafusion/labels/sql)] ([NGA-TRAN](https://github.com/NGA-TRAN))
- Enrich filter statistics with known column boundaries [\#4519](https://github.com/apache/datafusion/pull/4519) ([isidentical](https://github.com/isidentical))
- Remove Option from window frame [\#4516](https://github.com/apache/datafusion/pull/4516) [[sql](https://github.com/apache/datafusion/labels/sql)] ([mustafasrepo](https://github.com/mustafasrepo))
- Make nightly clippy happy [\#4515](https://github.com/apache/datafusion/pull/4515) [[sql](https://github.com/apache/datafusion/labels/sql)] ([xudong963](https://github.com/xudong963))
- Remove interior mutability of `MemTable` [\#4514](https://github.com/apache/datafusion/pull/4514) ([xudong963](https://github.com/xudong963))
- Make window function related struct public for ballista. [\#4511](https://github.com/apache/datafusion/pull/4511) ([Ted-Jiang](https://github.com/Ted-Jiang))
- minor: rename `push_down_limit` [\#4510](https://github.com/apache/datafusion/pull/4510) ([jackwener](https://github.com/jackwener))
- Add get_window_frame in window_expr, show frame info in window_agg_exec [\#4508](https://github.com/apache/datafusion/pull/4508) ([Ted-Jiang](https://github.com/Ted-Jiang))
- Add sqllogictest auto labeler [\#4506](https://github.com/apache/datafusion/pull/4506) ([mvanschellebeeck](https://github.com/mvanschellebeeck))
- Add some more aggregate sqllogictests and remove rust tests [\#4505](https://github.com/apache/datafusion/pull/4505) ([mvanschellebeeck](https://github.com/mvanschellebeeck))
- Remove sqllogictests CI run [\#4504](https://github.com/apache/datafusion/pull/4504) ([mvanschellebeeck](https://github.com/mvanschellebeeck))
- Refactor code for `insert` in sqllogictest [\#4503](https://github.com/apache/datafusion/pull/4503) ([xudong963](https://github.com/xudong963))
- Add empty string normalization to sqllogictests [\#4501](https://github.com/apache/datafusion/pull/4501) ([alamb](https://github.com/alamb))
- sqllogictest: A logging and command line filter [\#4497](https://github.com/apache/datafusion/pull/4497) ([alamb](https://github.com/alamb))
- Support `insert into` statement in sqllogictest [\#4496](https://github.com/apache/datafusion/pull/4496) ([xudong963](https://github.com/xudong963))
- Improve error handling for array downcasting [\#4493](https://github.com/apache/datafusion/pull/4493) ([retikulum](https://github.com/retikulum))
- Unify most of `SessionConfig` settings into `ConfigOptions` [\#4492](https://github.com/apache/datafusion/pull/4492) ([alamb](https://github.com/alamb))
- feat: support prepare statement [\#4490](https://github.com/apache/datafusion/pull/4490) [[sql](https://github.com/apache/datafusion/labels/sql)] ([NGA-TRAN](https://github.com/NGA-TRAN))
- Minor: Update docstrings and comments to aggregate code [\#4489](https://github.com/apache/datafusion/pull/4489) ([alamb](https://github.com/alamb))
- Fix panic in median "AggregateState is not a scalar aggregate" [\#4488](https://github.com/apache/datafusion/pull/4488) ([alamb](https://github.com/alamb))
- fix `push_down_projection` push redundant columns. [\#4487](https://github.com/apache/datafusion/pull/4487) ([jackwener](https://github.com/jackwener))
- Add window func related logic plan to proto ability. [\#4485](https://github.com/apache/datafusion/pull/4485) ([Ted-Jiang](https://github.com/Ted-Jiang))
- fix `Planner` don't generate `SubqueryAlias` and generate duplicated `SubqueryAlias` [\#4484](https://github.com/apache/datafusion/pull/4484) [[sql](https://github.com/apache/datafusion/labels/sql)] ([jackwener](https://github.com/jackwener))
- Improve parquet partition_file output display [\#4467](https://github.com/apache/datafusion/pull/4467) ([alamb](https://github.com/alamb))
- minor: remove redundant `unwrap()` [\#4463](https://github.com/apache/datafusion/pull/4463) ([jackwener](https://github.com/jackwener))
- Fix `Cte` in `from` clause with duplicated cte name [\#4461](https://github.com/apache/datafusion/pull/4461) [[sql](https://github.com/apache/datafusion/labels/sql)] ([xudong963](https://github.com/xudong963))
- Replace `&Option<T>` with `Option<&T>` part 2 [\#4458](https://github.com/apache/datafusion/pull/4458) ([askoa](https://github.com/askoa))
- Fix output_partitioning\(\), output_ordering\(\), equivalence_properties\(\) in WindowAggExec, shift the Column indexes [\#4455](https://github.com/apache/datafusion/pull/4455) ([mingmwang](https://github.com/mingmwang))
- fix `push_down_filter` for pushing filters on grouping columns rather than aggregate columns [\#4447](https://github.com/apache/datafusion/pull/4447) ([jackwener](https://github.com/jackwener))
- Add support for non-column key for equijoin when eliminating cross join to inner join [\#4443](https://github.com/apache/datafusion/pull/4443) [[sql](https://github.com/apache/datafusion/labels/sql)] ([ygf11](https://github.com/ygf11))
- Remove the schema checking when creating `CrossJoinExec` [\#4432](https://github.com/apache/datafusion/pull/4432) ([HaoYang670](https://github.com/HaoYang670))
- `date_part` support fractions of second [\#4385](https://github.com/apache/datafusion/pull/4385) ([comphead](https://github.com/comphead))
- Minor: use upstream RowSelection code from arrow `intersect_row_selection` [\#4340](https://github.com/apache/datafusion/pull/4340) ([alamb](https://github.com/alamb))
- Support type coercion for timestamp and utf8 [\#4312](https://github.com/apache/datafusion/pull/4312) ([andre-cc-natzka](https://github.com/andre-cc-natzka))
