<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

  http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
-->

# Apache DataFusion 46.0.0 Changelog

This release consists of 288 commits from 79 contributors. See credits at the end of this changelog for more information.

Please see the [Upgrade Guide] for help updating to DataFusion `46.0.0`

[upgrade guide]: https://datafusion.apache.org/library-user-guide/upgrading.html#datafusion-46-0-0

**Breaking changes:**

- bug: Fix NULL handling in array_slice, introduce `NullHandling` enum to `Signature` [#14289](https://github.com/apache/datafusion/pull/14289) (jkosh44)
- Update REGEXP_MATCH scalar function to support Utf8View [#14449](https://github.com/apache/datafusion/pull/14449) (Omega359)
- Introduce unified `DataSourceExec` for provided datasources, remove `ParquetExec`, `CsvExec`, etc [#14224](https://github.com/apache/datafusion/pull/14224) (mertak-synnada)
- Fix: Avoid recursive external error wrapping [#14371](https://github.com/apache/datafusion/pull/14371) (getChan)
- Add `DataFusionError::Collection` to return multiple `DataFusionError`s [#14439](https://github.com/apache/datafusion/pull/14439) (eliaperantoni)
- function: Allow more expressive array signatures [#14532](https://github.com/apache/datafusion/pull/14532) (jkosh44)
- feat: add resolved `target` to `DmlStatement` (to eliminate need for table lookup after deserialization) [#14631](https://github.com/apache/datafusion/pull/14631) (milenkovicm)
- Signature::Coercible with user defined implicit casting [#14440](https://github.com/apache/datafusion/pull/14440) (jayzhan211)
- Remove CountWildcardRule in Analyzer and move the functionality in ExprPlanner, add `plan_aggregate` and `plan_window` to planner [#14689](https://github.com/apache/datafusion/pull/14689) (jayzhan211)
- Simplify `FileSource::create_file_opener`'s signature [#14798](https://github.com/apache/datafusion/pull/14798) (AdamGS)
- StatisticsV2: initial statistics framework redesign [#14699](https://github.com/apache/datafusion/pull/14699) (Fly-Style)
- fix(substrait): Do not add implicit groupBy expressions in `LogicalPlanBuilder` or when building logical plans from Substrait [#14860](https://github.com/apache/datafusion/pull/14860) (anlinc)

**Performance related:**

- perf: Improve `median` with no grouping by 2X [#14399](https://github.com/apache/datafusion/pull/14399) (2010YOUY01)
- Improve performance 10%-100% in `FIRST_VALUE` / `LAST_VALUE` by not sort rows in `FirstValueAccumulator` [#14402](https://github.com/apache/datafusion/pull/14402) (blaginin)
- Speed up `uuid` UDF (40x faster) [#14675](https://github.com/apache/datafusion/pull/14675) (simonvandel)
- script to export benchmark information as Line Protocol format [#14662](https://github.com/apache/datafusion/pull/14662) (logan-keede)
- perf: Drop RowConverter from GroupOrderingPartial [#14566](https://github.com/apache/datafusion/pull/14566) (ctsk)
- Speedup `to_hex` (~2x faster) [#14686](https://github.com/apache/datafusion/pull/14686) (simonvandel)

**Implemented enhancements:**

- feat: Speed up `struct` and `named_struct` using `invoke_with_args` [#14276](https://github.com/apache/datafusion/pull/14276) (pepijnve)
- feat: add hint for missing fields [#14521](https://github.com/apache/datafusion/pull/14521) (Lordworms)
- feat: Add support for --mem-pool-type and --memory-limit options to multiple benchmarks [#14642](https://github.com/apache/datafusion/pull/14642) (Kontinuation)
- feat: Implement UNION ALL BY NAME [#14538](https://github.com/apache/datafusion/pull/14538) (rkrishn7)
- feat: Add ScalarUDF support in FFI crate [#14579](https://github.com/apache/datafusion/pull/14579) (timsaucer)
- feat: Improve datafusion-cli memory usage and considering reserve mem… [#14766](https://github.com/apache/datafusion/pull/14766) (zhuqi-lucas)

**Fixed bugs:**

- fix: Limits are not applied correctly [#14418](https://github.com/apache/datafusion/pull/14418) (zhuqi-lucas)
- fix(ci): build error with wasm [#14494](https://github.com/apache/datafusion/pull/14494) (Lordworms)
- fix(doc): remove AWS_PROFILE from supported S3 configuration [#14492](https://github.com/apache/datafusion/pull/14492) (hussein-awala)
- fix: `List` of `FixedSizeList` coercion issue in SQL [#14468](https://github.com/apache/datafusion/pull/14468) (alan910127)
- fix: order by expr rewrite fix [#14486](https://github.com/apache/datafusion/pull/14486) (akoshchiy)
- fix: rewrite fetch, skip of the Limit node in correct order [#14496](https://github.com/apache/datafusion/pull/14496) (evenyag)
- fix: Capture nullability in `Values` node planning [#14472](https://github.com/apache/datafusion/pull/14472) (rkrishn7)
- fix: case-sensitive quoted identifiers in DELETE statements [#14584](https://github.com/apache/datafusion/pull/14584) (nantunes)
- fix: Substrait serializer clippy error: not calling truncate [#14723](https://github.com/apache/datafusion/pull/14723) (niebayes)
- fix: normalize column names in table constraints [#14794](https://github.com/apache/datafusion/pull/14794) (jonahgao)
- fix: we are missing the unlimited case for bounded streaming when usi… [#14815](https://github.com/apache/datafusion/pull/14815) (zhuqi-lucas)
- fix: use `return_type_from_args` and mark nullable if any of the input is nullable [#14841](https://github.com/apache/datafusion/pull/14841) (rluvaton)

**Documentation updates:**

- Add related source code locations to errors [#13664](https://github.com/apache/datafusion/pull/13664) (eliaperantoni)
- docs: Fix create_udf examples [#14405](https://github.com/apache/datafusion/pull/14405) (nuno-faria)
- Script and documentation for regenerating sqlite test files [#14290](https://github.com/apache/datafusion/pull/14290) (Omega359)
- Improve documentation about extended tests [#14320](https://github.com/apache/datafusion/pull/14320) (alamb)
- Add `Cargo.lock` [#14483](https://github.com/apache/datafusion/pull/14483) (mbrobbel)
- Test all examples from library-user-guide & user-guide docs [#14544](https://github.com/apache/datafusion/pull/14544) (ugoa)
- Add guideline for GSoC 2025 applicants under Contributor Guide [#14582](https://github.com/apache/datafusion/pull/14582) (oznur-synnada)
- Fix typo in comments [#14605](https://github.com/apache/datafusion/pull/14605) (byte-sourcerer)
- Update Community Events in concepts-readings-events.md [#14629](https://github.com/apache/datafusion/pull/14629) (oznur-synnada)
- Minor: Add docs and examples for `DataFusionErrorBuilder` [#14551](https://github.com/apache/datafusion/pull/14551) (alamb)
- docs: Add Sleeper to list of known users [#14648](https://github.com/apache/datafusion/pull/14648) (m09526)
- Add documentation for prepare statements. [#14639](https://github.com/apache/datafusion/pull/14639) (dhegberg)
- Update features / status documentation page [#14645](https://github.com/apache/datafusion/pull/14645) (alamb)
- Add union_extract scalar function [#12116](https://github.com/apache/datafusion/pull/12116) (gstvg)
- Fix CI doctests on main [#14667](https://github.com/apache/datafusion/pull/14667) (alamb)
- chore: adding Linkedin follow page [#14676](https://github.com/apache/datafusion/pull/14676) (comphead)
- Improve EnforceSorting docs. [#14673](https://github.com/apache/datafusion/pull/14673) (wiedld)
- Specify rust toolchain explicitly, document how to change it [#14655](https://github.com/apache/datafusion/pull/14655) (alamb)
- Create gsoc_project_ideas.md [#14774](https://github.com/apache/datafusion/pull/14774) (oznur-synnada)
- docs: Add additional info about memory reservation to the doc of MemoryPool [#14789](https://github.com/apache/datafusion/pull/14789) (Kontinuation)
- docs: Add instruction to build [#14694](https://github.com/apache/datafusion/pull/14694) (dentiny)
- Update website links [#14846](https://github.com/apache/datafusion/pull/14846) (oznur-synnada)
- Improve benchmark docs [#14820](https://github.com/apache/datafusion/pull/14820) (carols10cents)
- Add polygon.io to user list [#14871](https://github.com/apache/datafusion/pull/14871) (xudong963)
- Update dft in intro "Known Users" [#14875](https://github.com/apache/datafusion/pull/14875) (matthewmturner)
- Add `statistics_truncate_length` parquet writer config [#14782](https://github.com/apache/datafusion/pull/14782) (akoshchiy)
- minor: Update docs and error messages about what SQL dialects are supported [#14893](https://github.com/apache/datafusion/pull/14893) (AdamGS)
- Minor: Add Development Environment to Documentation Index [#14890](https://github.com/apache/datafusion/pull/14890) (alamb)
- Examples: boundary analysis example for `AND/OR` conjunctions [#14735](https://github.com/apache/datafusion/pull/14735) (clflushopt)
- Allow setting the recursion limit for sql parsing [#14756](https://github.com/apache/datafusion/pull/14756) (cetra3)
- Document SQL literal syntax and escaping [#14934](https://github.com/apache/datafusion/pull/14934) (alamb)
- Prepare for 46.0.0 release: Version and Changelog [#14903](https://github.com/apache/datafusion/pull/14903) (xudong963)
- MINOR fix(docs): set the proper link for dev-env setup in contrib guide [#14960](https://github.com/apache/datafusion/pull/14960) (clflushopt)

**Other:**

- Fix join type coercion when joining 2 relations with the same name via `DataFrame` API [#14387](https://github.com/apache/datafusion/pull/14387) (alamb)
- Minor: fix typo in test name [#14403](https://github.com/apache/datafusion/pull/14403) (alamb)
- test: add regression test for unnesting dictionary encoded columns [#14395](https://github.com/apache/datafusion/pull/14395) (duongcongtoai)
- chore: Upgrade to `arrow`/`parquet` `54.1.0` and fix clippy/ci [#14415](https://github.com/apache/datafusion/pull/14415) (Weijun-H)
- bump arrow version to 54.1.0 and fix clippy error [#14414](https://github.com/apache/datafusion/pull/14414) (Lordworms)
- Support `array_concat` for `Utf8View` [#14378](https://github.com/apache/datafusion/pull/14378) (alamb)
- Fully support LIKE/ILIKE with Utf8View [#14379](https://github.com/apache/datafusion/pull/14379) (alamb)
- Fix `null` input in `map_keys/values` [#14401](https://github.com/apache/datafusion/pull/14401) (cht42)
- Remove dependency on datafusion_catalog from datafusion-cli [#14398](https://github.com/apache/datafusion/pull/14398) (alamb)
- chore(deps): update substrait requirement from 0.52 to 0.53 [#14419](https://github.com/apache/datafusion/pull/14419) (dependabot[bot])
- move resolve_table_references`out of`datafusion-catalog` [#14441](https://github.com/apache/datafusion/pull/14441) (logan-keede)
- Fix a clippy warning [#14445](https://github.com/apache/datafusion/pull/14445) (mbrobbel)
- Resolve a todo about using workspace dependencies [#14443](https://github.com/apache/datafusion/pull/14443) (mbrobbel)
- Support `Utf8View` to `numeric` coercion [#14377](https://github.com/apache/datafusion/pull/14377) (alamb)
- Fix regression list Type Coercion List with inner type struct which has large/view types [#14385](https://github.com/apache/datafusion/pull/14385) (alamb)
- Improve error message on unsupported correlation [#14458](https://github.com/apache/datafusion/pull/14458) (findepi)
- Replace `once_cell::Lazy` with `std::sync::LazyLock` [#14480](https://github.com/apache/datafusion/pull/14480) (mbrobbel)
- chore(deps): bump bytes from 1.9.0 to 1.10.0 in /datafusion-cli [#14476](https://github.com/apache/datafusion/pull/14476) (dependabot[bot])
- chore(deps): bump clap from 4.5.27 to 4.5.28 in /datafusion-cli [#14477](https://github.com/apache/datafusion/pull/14477) (dependabot[bot])
- chore: Fix link to issue and expand comment [#14473](https://github.com/apache/datafusion/pull/14473) (findepi)
- Make Pushdown Filters Public [#14471](https://github.com/apache/datafusion/pull/14471) (cetra3)
- Minor: `cargo fmt` to fix CI [#14487](https://github.com/apache/datafusion/pull/14487) (alamb)
- chore: clean up dependencies for datafusion cli [#14484](https://github.com/apache/datafusion/pull/14484) (comphead)
- Provide user-defined invariants for logical node extensions. [#14329](https://github.com/apache/datafusion/pull/14329) (wiedld)
- DFParser should skip unsupported COPY INTO [#14382](https://github.com/apache/datafusion/pull/14382) (osipovartem)
- Improve Unparser (scalar_to_sql) to respect dialect timestamp type overrides [#14407](https://github.com/apache/datafusion/pull/14407) (sgrebnov)
- Fix link to volcano parallelism paper [#14497](https://github.com/apache/datafusion/pull/14497) (lewiszlw)
- chore(deps): bump aws-config from 1.5.15 to 1.5.16 in /datafusion-cli [#14500](https://github.com/apache/datafusion/pull/14500) (dependabot[bot])
- chore: Add more LIKE with escape tests [#14501](https://github.com/apache/datafusion/pull/14501) (findepi)
- Fix a clippy warning in `datafusion-sqllogictest` [#14506](https://github.com/apache/datafusion/pull/14506) (mbrobbel)
- minor: improve PR template [#14507](https://github.com/apache/datafusion/pull/14507) (alamb)
- Support `Dictionary` and `List` types in `scalar_to_sql` [#14346](https://github.com/apache/datafusion/pull/14346) (cetra3)
- Serialize `parquet_options` in `datafusion-proto` [#14465](https://github.com/apache/datafusion/pull/14465) (blaginin)
- make datafusion-catalog-listing and move some implementation of listing out of datafusion/core/datasource/listing [#14464](https://github.com/apache/datafusion/pull/14464) (logan-keede)
- refactor: remove uses of `arrow_buffer` & `arrow_array` and use reexport in arrow instead [#14503](https://github.com/apache/datafusion/pull/14503) (Chen-Yuan-Lai)
- core: Support uncorrelated EXISTS [#14474](https://github.com/apache/datafusion/pull/14474) (findepi)
- chore(deps): Update sqlparser to `0.54.0` [#14255](https://github.com/apache/datafusion/pull/14255) (alamb)
- Validate and unpack function arguments tersely [#14513](https://github.com/apache/datafusion/pull/14513) (findepi)
- bug: Fix edge cases in array_slice [#14489](https://github.com/apache/datafusion/pull/14489) (jkosh44)
- Feat: Add fetch to CoalescePartitionsExec [#14499](https://github.com/apache/datafusion/pull/14499) (mertak-synnada)
- Improve error messages to include the function name. [#14511](https://github.com/apache/datafusion/pull/14511) (Omega359)
- to_unixtime does not support timestamps with a timezone [#14490](https://github.com/apache/datafusion/pull/14490) (Omega359)
- bug: Remove array_slice two arg variant [#14527](https://github.com/apache/datafusion/pull/14527) (jkosh44)
- Minor: deprecate unused index mod [#14534](https://github.com/apache/datafusion/pull/14534) (zhuqi-lucas)
- Fix config_namespace macro symbol usage [#14520](https://github.com/apache/datafusion/pull/14520) (davisp)
- functions: Remove NullHandling from scalar funcs [#14531](https://github.com/apache/datafusion/pull/14531) (jkosh44)
- Relax physical schema validation [#14519](https://github.com/apache/datafusion/pull/14519) (findepi)
- Minor: Update changelog for `45.0.0` and tweak `CHANGELOG` docs [#14545](https://github.com/apache/datafusion/pull/14545) (alamb)
- minor: polish MemoryStream related code [#14537](https://github.com/apache/datafusion/pull/14537) (zjregee)
- refactor: switch BooleanBufferBuilder to NullBufferBuilder in MaybeNullBufferBuilder [#14504](https://github.com/apache/datafusion/pull/14504) (Chen-Yuan-Lai)
- Allow constructing ScalarUDF from shared implementation [#14541](https://github.com/apache/datafusion/pull/14541) (findepi)
- some dependency removals and setup for refactor of `FileScanConfig` [#14543](https://github.com/apache/datafusion/pull/14543) (logan-keede)
- Always use `StringViewArray` as output of `substr` [#14498](https://github.com/apache/datafusion/pull/14498) (Kev1n8)
- refactor: remove remaining uses of `arrow_array` and use reexport in `arrow` instead [#14528](https://github.com/apache/datafusion/pull/14528) (Chen-Yuan-Lai)
- chore: update datafusion-testing pin to fix extended tests [#14556](https://github.com/apache/datafusion/pull/14556) (alamb)
- chore: remove partition_keys from (Bounded)WindowAggExec [#14526](https://github.com/apache/datafusion/pull/14526) (irenjj)
- chore(deps): bump nix from 0.28.0 to 0.29.0 [#14559](https://github.com/apache/datafusion/pull/14559) (dependabot[bot])
- use a single row_count column during predicate pruning instead of one per column [#14295](https://github.com/apache/datafusion/pull/14295) (adriangb)
- Update proto to support to/from json with an extension codec [#14561](https://github.com/apache/datafusion/pull/14561) (Omega359)
- Remove useless test util [#14570](https://github.com/apache/datafusion/pull/14570) (xudong963)
- minor: Move file compression to `datafusion-catalog-listing` [#14555](https://github.com/apache/datafusion/pull/14555) (logan-keede)
- chore(deps): bump strum from 0.26.3 to 0.27.0 [#14573](https://github.com/apache/datafusion/pull/14573) (dependabot[bot])
- Minor: remove unnecessary dependencies in `datafusion-sqllogictest` [#14578](https://github.com/apache/datafusion/pull/14578) (alamb)
- Fix: limit is missing after removing SPM [#14569](https://github.com/apache/datafusion/pull/14569) (xudong963)
- Adding cargo clean at the end of every step [#14592](https://github.com/apache/datafusion/pull/14592) (Omega359)
- Make it easier to create a ScalarValure representing typed null (#14548) [#14558](https://github.com/apache/datafusion/pull/14558) (cj-zhukov)
- chore(deps): bump substrait from 0.53.0 to 0.53.1 [#14599](https://github.com/apache/datafusion/pull/14599) (dependabot[bot])
- refactor: remove uses of arrow_schema and use reexport in arrow instead [#14597](https://github.com/apache/datafusion/pull/14597) (Chen-Yuan-Lai)
- Benchmark showcasing with_column and with_column_renamed function performance [#14564](https://github.com/apache/datafusion/pull/14564) (Omega359)
- Remove use of deprecated dict_id in datafusion-proto (#14173) [#14227](https://github.com/apache/datafusion/pull/14227) (cj-zhukov)
- refactor: Move FileSinkConfig out of Core [#14585](https://github.com/apache/datafusion/pull/14585) (logan-keede)
- Revert modification of build dependency [#14606](https://github.com/apache/datafusion/pull/14606) (ugoa)
- chore(deps): bump serialize-javascript and copy-webpack-plugin in /datafusion/wasmtest/datafusion-wasm-app [#14594](https://github.com/apache/datafusion/pull/14594) (dependabot[bot])
- cli: Add nested expressions [#14614](https://github.com/apache/datafusion/pull/14614) (jkosh44)
- Minor: remove some unnecessary dependencies [#14615](https://github.com/apache/datafusion/pull/14615) (logan-keede)
- Disable extended tests (`extended_tests`) that are failing on runner [#14604](https://github.com/apache/datafusion/pull/14604) (alamb)
- minor: check size overflow before string repeat build [#14575](https://github.com/apache/datafusion/pull/14575) (wForget)
- Speedup `date_trunc` (~20% time reduction) [#14593](https://github.com/apache/datafusion/pull/14593) (simonvandel)
- equivalence classes: use normalized mapping for projection [#14327](https://github.com/apache/datafusion/pull/14327) (askalt)
- chore(deps): bump prost-build from 0.13.4 to 0.13.5 [#14623](https://github.com/apache/datafusion/pull/14623) (dependabot[bot])
- chore(deps): bump bzip2 from 0.5.0 to 0.5.1 [#14620](https://github.com/apache/datafusion/pull/14620) (dependabot[bot])
- chore(deps): bump clap from 4.5.28 to 4.5.29 [#14619](https://github.com/apache/datafusion/pull/14619) (dependabot[bot])
- chore(deps): bump sqllogictest from 0.26.4 to 0.27.0 [#14598](https://github.com/apache/datafusion/pull/14598) (dependabot[bot])
- Fix ci test [#14625](https://github.com/apache/datafusion/pull/14625) (xudong963)
- chore(deps): group `prost` and `pbjson` dependabot updates [#14626](https://github.com/apache/datafusion/pull/14626) (mbrobbel)
- chore(deps): bump substrait from 0.53.1 to 0.53.2 [#14627](https://github.com/apache/datafusion/pull/14627) (dependabot[bot])
- refactor: Move various parts of datasource out of core [#14616](https://github.com/apache/datafusion/pull/14616) (logan-keede)
- Use ` take_function_args` in more places [#14525](https://github.com/apache/datafusion/pull/14525) (lgingerich)
- Minor: remove unused `AutoFinishBzEncoder` [#14630](https://github.com/apache/datafusion/pull/14630) (jonahgao)
- Add test for nullable doesn't work when create memory table [#14624](https://github.com/apache/datafusion/pull/14624) (xudong963)
- Fallback to Utf8View for `Dict(_, Utf8View)` in `type_union_resolution_coercion` [#14602](https://github.com/apache/datafusion/pull/14602) (jayzhan211)
- refactor: Make catalog datasource [#14643](https://github.com/apache/datafusion/pull/14643) (logan-keede)
- Implement predicate pruning for not like expressions [#14567](https://github.com/apache/datafusion/pull/14567) (UBarney)
- Migrate math functions to implement invoke_with_args [#14658](https://github.com/apache/datafusion/pull/14658) (lewiszlw)
- bug: fix offset type mismatch when prepending lists [#14672](https://github.com/apache/datafusion/pull/14672) (friendlymatthew)
- Minor: remove confusing `update_plan_from_children` call from `EnforceSorting` [#14650](https://github.com/apache/datafusion/pull/14650) (xudong963)
- Improve UX Rename `FileScanConfig::new_exec` to `FileScanConfig::build` [#14670](https://github.com/apache/datafusion/pull/14670) (alamb)
- Consolidate and expand ident normalization tests [#14374](https://github.com/apache/datafusion/pull/14374) (alamb)
- Update GitHub CI run image for license check job [#14674](https://github.com/apache/datafusion/pull/14674) (findepi)
- Allow extensions_options to accept Option field [#14664](https://github.com/apache/datafusion/pull/14664) (goldmedal)
- Minor: Re-export `datafusion_expr_common` crate [#14696](https://github.com/apache/datafusion/pull/14696) (jayzhan211)
- Migrate the internal and testing functions to invoke_with_args [#14693](https://github.com/apache/datafusion/pull/14693) (goldmedal)
- Improve docs `TableSource` and `DefaultTableSource` [#14665](https://github.com/apache/datafusion/pull/14665) (alamb)
- Improve SQL Planner docs [#14669](https://github.com/apache/datafusion/pull/14669) (alamb)
- MIgrate math function macro to implement invoke_with_args [#14690](https://github.com/apache/datafusion/pull/14690) (goldmedal)
- Improve `downcast_value!` macro [#14683](https://github.com/apache/datafusion/pull/14683) (findepi)
- chore(deps): bump tempfile from 3.16.0 to 3.17.0 [#14713](https://github.com/apache/datafusion/pull/14713) (dependabot[bot])
- bug: improve schema checking for `insert into` cases [#14572](https://github.com/apache/datafusion/pull/14572) (zhuqi-lucas)
- Early exit on column normalisation to improve DataFrame performance [#14636](https://github.com/apache/datafusion/pull/14636) (blaginin)
- Add example for `LogicalPlanBuilder::insert_into` [#14663](https://github.com/apache/datafusion/pull/14663) (alamb)
- optimize performance of the repeat function (up to 50% faster) [#14697](https://github.com/apache/datafusion/pull/14697) (zjregee)
- `AggregateUDFImpl::schema_name` and `AggregateUDFImpl::display_name` for customizable name [#14695](https://github.com/apache/datafusion/pull/14695) (jayzhan211)
- Add an example of boundary analysis simple expressions. [#14688](https://github.com/apache/datafusion/pull/14688) (clflushopt)
- chore(deps): bump arrow-ipc from 54.1.0 to 54.2.0 [#14719](https://github.com/apache/datafusion/pull/14719) (dependabot[bot])
- chore(deps): bump strum from 0.27.0 to 0.27.1 [#14718](https://github.com/apache/datafusion/pull/14718) (dependabot[bot])
- minor: enable decimal dictionary sbbf pruning test [#14711](https://github.com/apache/datafusion/pull/14711) (korowa)
- chore(deps): bump sqllogictest from 0.27.0 to 0.27.1 [#14717](https://github.com/apache/datafusion/pull/14717) (dependabot[bot])
- minor: simplify `union_extract` code [#14640](https://github.com/apache/datafusion/pull/14640) (alamb)
- make DefaultSubstraitProducer public [#14721](https://github.com/apache/datafusion/pull/14721) (gabotechs)
- chore: Migrate Encoding functions to invoke_with_args [#14727](https://github.com/apache/datafusion/pull/14727) (irenjj)
- chore: Migrate Core Functions to invoke_with_args [#14725](https://github.com/apache/datafusion/pull/14725) (niebayes)
- Fix off by 1 in decimal cast to lower precision [#14731](https://github.com/apache/datafusion/pull/14731) (findepi)
- migrate string functions to `inovke_with_args` [#14722](https://github.com/apache/datafusion/pull/14722) (zjregee)
- chore: Migrate Array Functions to invoke_with_args [#14726](https://github.com/apache/datafusion/pull/14726) (irenjj)
- chore: Migrate Regex function to invoke_with_args [#14728](https://github.com/apache/datafusion/pull/14728) (irenjj)
- bug: Fix memory reservation and allocation problems for SortExec [#14644](https://github.com/apache/datafusion/pull/14644) (Kontinuation)
- Skip target in taplo checks [#14747](https://github.com/apache/datafusion/pull/14747) (findepi)
- chore(deps): bump uuid from 1.13.1 to 1.13.2 [#14739](https://github.com/apache/datafusion/pull/14739) (dependabot[bot])
- chore(deps): bump blake3 from 1.5.5 to 1.6.0 [#14741](https://github.com/apache/datafusion/pull/14741) (dependabot[bot])
- chore(deps): bump tempfile from 3.17.0 to 3.17.1 [#14742](https://github.com/apache/datafusion/pull/14742) (dependabot[bot])
- chore(deps): bump clap from 4.5.29 to 4.5.30 [#14743](https://github.com/apache/datafusion/pull/14743) (dependabot[bot])
- chore(deps): bump parquet from 54.1.0 to 54.2.0 [#14744](https://github.com/apache/datafusion/pull/14744) (dependabot[bot])
- Speed up `chr` UDF (~4x faster) [#14700](https://github.com/apache/datafusion/pull/14700) (simonvandel)
- Support aliases in ConstEvaluator [#14734](https://github.com/apache/datafusion/pull/14734) (joroKr21)
- `AggregateUDFImpl::window_function_schema_name` and `AggregateUDFImpl::window_function_display_name` for window aggregate function [#14750](https://github.com/apache/datafusion/pull/14750) (jayzhan211)
- chore: migrate crypto functions to invoke_with_args [#14764](https://github.com/apache/datafusion/pull/14764) (Chen-Yuan-Lai)
- minor: remove custom extract_ok! macro [#14733](https://github.com/apache/datafusion/pull/14733) (ctsk)
- Minor: Further Clean-up in Enforce Sorting [#14732](https://github.com/apache/datafusion/pull/14732) (berkaysynnada)
- chore(deps): bump arrow-flight from 54.1.0 to 54.2.0 [#14786](https://github.com/apache/datafusion/pull/14786) (dependabot[bot])
- chore(deps): bump serde_json from 1.0.138 to 1.0.139 [#14784](https://github.com/apache/datafusion/pull/14784) (dependabot[bot])
- dependabot: group arrow/parquet minor/patch bumps, remove limit [#14730](https://github.com/apache/datafusion/pull/14730) (mbrobbel)
- Map access supports constant-resolvable expressions [#14712](https://github.com/apache/datafusion/pull/14712) (Lordworms)
- Fix build after logical conflict [#14791](https://github.com/apache/datafusion/pull/14791) (alamb)
- Fix CI job test-datafusion-pyarrow [#14790](https://github.com/apache/datafusion/pull/14790) (Owen-CH-Leung)
- Use `doc_auto_cfg`, logo and favicon for docs.rs [#14746](https://github.com/apache/datafusion/pull/14746) (mbrobbel)
- chore(deps): bump sqllogictest from 0.27.1 to 0.27.2 [#14785](https://github.com/apache/datafusion/pull/14785) (dependabot[bot])
- Fix CI fail for extended test (by freeing up more disk space in CI runner) [#14745](https://github.com/apache/datafusion/pull/14745) (2010YOUY01)
- chore: Benchmark deps cleanup [#14793](https://github.com/apache/datafusion/pull/14793) (findepi)
- chore: Fix test not to litter in repository [#14795](https://github.com/apache/datafusion/pull/14795) (findepi)
- chore(deps): bump testcontainers from 0.23.2 to 0.23.3 [#14787](https://github.com/apache/datafusion/pull/14787) (dependabot[bot])
- chore(deps): bump serde from 1.0.217 to 1.0.218 [#14788](https://github.com/apache/datafusion/pull/14788) (dependabot[bot])
- refactor: move `DataSource` to `datafusion-datasource` [#14671](https://github.com/apache/datafusion/pull/14671) (logan-keede)
- Fix Clippy 1.85 warnings [#14800](https://github.com/apache/datafusion/pull/14800) (mbrobbel)
- Allow `FileSource`-specific repartitioning [#14754](https://github.com/apache/datafusion/pull/14754) (AdamGS)
- Bump MSRV to 1.82, toolchain to 1.85 [#14811](https://github.com/apache/datafusion/pull/14811) (mbrobbel)
- Chore/Add additional FFI unit tests [#14802](https://github.com/apache/datafusion/pull/14802) (timsaucer)
- Minor: comment in Cargo.toml about MSRV [#14809](https://github.com/apache/datafusion/pull/14809) (alamb)
- Remove unused crate dependencies [#14827](https://github.com/apache/datafusion/pull/14827) (findepi)
- fix(physical-expr): Remove empty constants check when ordering is satisfied [#14829](https://github.com/apache/datafusion/pull/14829) (rkrishn7)
- chore(deps): bump log from 0.4.25 to 0.4.26 [#14847](https://github.com/apache/datafusion/pull/14847) (dependabot[bot])
- Minor: Ignore examples output directory [#14840](https://github.com/apache/datafusion/pull/14840) (AdamGS)
- Add support for `Dictionary` to AST datatype in unparser [#14783](https://github.com/apache/datafusion/pull/14783) (cetra3)
- Add `range` table function [#14830](https://github.com/apache/datafusion/pull/14830) (simonvandel)
- chore: migrate invoke_batch to invoke_with_args for unicode function [#14856](https://github.com/apache/datafusion/pull/14856) (onlyjackfrost)
- test: change test_function macro to use `return_type_from_args` instead of `return_type` [#14852](https://github.com/apache/datafusion/pull/14852) (rluvaton)
- Move `FileSourceConfig` and `FileStream` to the new `datafusion-datasource` [#14838](https://github.com/apache/datafusion/pull/14838) (AdamGS)
- Minor: Counting elapsed_compute in BoundedWindowAggExec [#14869](https://github.com/apache/datafusion/pull/14869) (2010YOUY01)
- Optimize `gcd` for array and scalar case by avoiding `make_scalar_function` where has unnecessary conversion between scalar and array [#14834](https://github.com/apache/datafusion/pull/14834) (jayzhan211)
- refactor: replace OnceLock with LazyLock [#14870](https://github.com/apache/datafusion/pull/14870) (AmosAidoo)
- Workaround for compilation error due to rkyv#434. [#14863](https://github.com/apache/datafusion/pull/14863) (ryzhyk)
- chore(deps): bump uuid from 1.13.2 to 1.14.0 [#14866](https://github.com/apache/datafusion/pull/14866) (dependabot[bot])
- refactor: replace OnceLock with LazyLock [#14880](https://github.com/apache/datafusion/pull/14880) (AmosAidoo)
- chore: migrate to `invoke_with_args` for datetime functions [#14876](https://github.com/apache/datafusion/pull/14876) (onlyjackfrost)
- Fix `regenerate_sqlite_files.sh` due to changes in sqllogictests [#14881](https://github.com/apache/datafusion/pull/14881) (alamb)
- Move `FileFormat` and related pieces to `datafusion-datasource` [#14873](https://github.com/apache/datafusion/pull/14873) (AdamGS)
- fix duplicated schema name error from count wildcard [#14824](https://github.com/apache/datafusion/pull/14824) (jayzhan211)
- replace TypeSignature::String with TypeSignature::Coercible for trim functions [#14865](https://github.com/apache/datafusion/pull/14865) (zjregee)
- Window Functions Order Conservation -- Follow-up On Set Monotonicity [#14813](https://github.com/apache/datafusion/pull/14813) (berkaysynnada)
- Implement builder style API for ParserOptions [#14887](https://github.com/apache/datafusion/pull/14887) (kosiew)
- chore: Attach Diagnostic to "function x does not exist" error [#14849](https://github.com/apache/datafusion/pull/14849) (onlyjackfrost)
- Fix: External sort failing on `StringView` due to shared buffers [#14823](https://github.com/apache/datafusion/pull/14823) (2010YOUY01)
- refactor: make SqlToRel::new derive the parser options from the context provider [#14822](https://github.com/apache/datafusion/pull/14822) (niebayes)
- Datafusion-cli: Redesign the datafusion-cli execution and print, make it totally streaming printing without memory overhead [#14877](https://github.com/apache/datafusion/pull/14877) (zhuqi-lucas)
- chore: Strip debuginfo symbols for release [#14843](https://github.com/apache/datafusion/pull/14843) (comphead)
- chore(deps): bump zstd from 0.13.2 to 0.13.3 [#14889](https://github.com/apache/datafusion/pull/14889) (dependabot[bot])
- Add DataFrame fill_null [#14769](https://github.com/apache/datafusion/pull/14769) (kosiew)
- Cancellation benchmark [#14818](https://github.com/apache/datafusion/pull/14818) (carols10cents)
- Include struct name on FileScanConfig debug impl [#14883](https://github.com/apache/datafusion/pull/14883) (alamb)
- Preserve the name of grouping sets in SimplifyExpressions [#14888](https://github.com/apache/datafusion/pull/14888) (joroKr21)
- Require `Debug` for `DataSource` [#14882](https://github.com/apache/datafusion/pull/14882) (alamb)
- Update regenerate sql dep, revert runner changes. [#14901](https://github.com/apache/datafusion/pull/14901) (Omega359)
- chore(deps): bump flate2 from 1.0.35 to 1.1.0 [#14848](https://github.com/apache/datafusion/pull/14848) (dependabot[bot])
- replace TypeSignature::String with TypeSignature::Coercible for starts_with [#14812](https://github.com/apache/datafusion/pull/14812) (zjregee)
- Dataframe with_column and with_column_renamed performance improvements [#14653](https://github.com/apache/datafusion/pull/14653) (Omega359)
- chore(deps): bump uuid from 1.14.0 to 1.15.1 [#14911](https://github.com/apache/datafusion/pull/14911) (dependabot[bot])
- chore(deps): bump libc from 0.2.169 to 0.2.170 [#14912](https://github.com/apache/datafusion/pull/14912) (dependabot[bot])
- Move HashJoin from `RawTable` to `HashTable` [#14904](https://github.com/apache/datafusion/pull/14904) (Dandandan)
- Rename `DataSource` and `FileSource` fields for consistency [#14898](https://github.com/apache/datafusion/pull/14898) (alamb)
- Fix the null handling for to_char function [#14908](https://github.com/apache/datafusion/pull/14908) (kosiew)
- Add tests for Demonstrate EnforceSorting can remove a needed coalesce [#14919](https://github.com/apache/datafusion/pull/14919) (wiedld)
- Fix: New Datafusion-cli streaming printing way should handle corner case for only one small batch which lines are less than max_rows [#14921](https://github.com/apache/datafusion/pull/14921) (zhuqi-lucas)
- Add docs to `update_coalesce_ctx_children`. [#14907](https://github.com/apache/datafusion/pull/14907) (wiedld)
- chore(deps): bump the arrow-parquet group with 7 updates [#14930](https://github.com/apache/datafusion/pull/14930) (dependabot[bot])
- chore(deps): bump aws-config from 1.5.16 to 1.5.17 [#14931](https://github.com/apache/datafusion/pull/14931) (dependabot[bot])
- Add additional protobuf tests for plans that read parquet with projections [#14924](https://github.com/apache/datafusion/pull/14924) (alamb)
- Fix link in datasource readme [#14928](https://github.com/apache/datafusion/pull/14928) (lewiszlw)
- Expose `build_row_filter` method [#14933](https://github.com/apache/datafusion/pull/14933) (xudong963)
- Do not unescape backslashes in datafusion-cli [#14844](https://github.com/apache/datafusion/pull/14844) (Lordworms)
- Set projection before configuring the source [#14685](https://github.com/apache/datafusion/pull/14685) (blaginin)
- Add H2O.ai Database-like Ops benchmark to dfbench (join support) [#14902](https://github.com/apache/datafusion/pull/14902) (zhuqi-lucas)
- Use arrow IPC Stream format for spill files [#14868](https://github.com/apache/datafusion/pull/14868) (davidhewitt)
- refactor(properties): Split properties.rs into smaller modules [#14925](https://github.com/apache/datafusion/pull/14925) (Standing-Man)
- Fix failing extended `sqlite`test on main / update `datafusion-testing` pin [#14940](https://github.com/apache/datafusion/pull/14940) (alamb)
- Revert Datafusion-cli: Redesign the datafusion-cli execution and print, make it totally streaming printing without memory overhead [#14948](https://github.com/apache/datafusion/pull/14948) (alamb)
- Remove invalid bug reproducer. [#14950](https://github.com/apache/datafusion/pull/14950) (wiedld)
- Improve documentation for `DataSourceExec`, `FileScanConfig`, `DataSource` etc [#14941](https://github.com/apache/datafusion/pull/14941) (alamb)
- Do not swap with projection when file is partitioned [#14956](https://github.com/apache/datafusion/pull/14956) (blaginin)
- Minor: Add more projection pushdown tests, clarify comments [#14963](https://github.com/apache/datafusion/pull/14963) (alamb)
- Update labeler components [#14942](https://github.com/apache/datafusion/pull/14942) (alamb)
- Deprecate `Expr::Wildcard` [#14959](https://github.com/apache/datafusion/pull/14959) (linhr)

## Credits

Thank you to everyone who contributed to this release. Here is a breakdown of commits (PRs merged) per contributor.

```
    38	Andrew Lamb
    35	dependabot[bot]
    14	Piotr Findeisen
    10	Matthijs Brobbel
    10	logan-keede
     9	Bruce Ritchie
     9	xudong.w
     8	Jay Zhan
     8	Qi Zhu
     6	Adam Gutglick
     6	Joseph Koshakow
     5	Ian Lai
     5	Lordworms
     5	Simon Vandel Sillesen
     5	wiedld
     5	zjregee
     4	Dmitrii Blaginin
     4	Kristin Cowalcijk
     4	Peter L
     4	Yongting You
     4	irenjj
     4	oznur-synnada
     3	Andy Yen
     3	Jax Liu
     3	Oleks V
     3	Rohan Krishnaswamy
     3	Tim Saucer
     3	kosiew
     3	niebayes
     3	张林伟
     2	@clflushopt
     2	Amos Aidoo
     2	Andrey Koshchiy
     2	Berkay Şahin
     2	Carol (Nichols || Goulding)
     2	Christian
     2	Dawei H.
     2	Elia Perantoni
     2	Georgi Krastev
     2	Jonah Gao
     2	Raz Luvaton
     2	Sergey Zhukov
     2	mertak-synnada
     1	Adrian Garcia Badaracco
     1	Alan Tang
     1	Albert Skalt
     1	Alex Huang
     1	Anlin Chen
     1	Artem Osipov
     1	Daniel Hegberg
     1	Daniël Heres
     1	David Hewitt
     1	Duong Cong Toai
     1	Eduard Karacharov
     1	Gabriel
     1	Hussein Awala
     1	Kaifeng Zheng
     1	Landon Gingerich
     1	Leonid Ryzhyk
     1	Li-Lun Lin
     1	Marko Milenković
     1	Matthew Kim
     1	Matthew Turner
     1	Namgung Chan
     1	Nelson Antunes
     1	Owen Leung
     1	Paul J. Davis
     1	Pepijn Van Eeckhoudt
     1	Sasha Syrotenko
     1	Sergei Grebnov
     1	UBarney
     1	Yingwen
     1	Zhen Wang
     1	cht42
     1	cjw
     1	dentiny
     1	gstvg
     1	m09526
     1	nuno-faria
```

Thank you also to everyone who contributed in other ways such as filing issues, reviewing PRs, and providing feedback on this release.
