← Back
Databricks serverless compute version 18.0 launches with SQL scripting GA and streaming query improvements
· releasefeatureapiperformance · docs.databricks.com ↗

Key Features in Version 18.0

Databricks serverless compute version 18.0 is now available, corresponding to Databricks Runtime 18.0. The release introduces several important capabilities across SQL, streaming, and data integration:

SQL Scripting and Advanced Functions

SQL scripting is now generally available, enabling users to write procedural SQL logic with control flow statements. The release also expands SQL functionality with new window function support in metric views, the FILTER clause for measure aggregate functions, and newly available functions like BITMAP_AND_AGG and Theta sketch functions for approximate distinct count operations.

Streaming and Query Optimization

Stateless streaming queries now support Adaptive Query Execution (AQE) and auto-optimized shuffle (AOS), improving query performance automatically. Additionally, you can now dynamically adjust the number of shuffle partitions in stateless streaming queries without restarting, providing greater flexibility for tuning production workloads.

Performance and Isolation Improvements

Shared isolation execution environments for Unity Catalog Python UDFs now allow UDFs with the same owner to share an isolation environment by default, reducing memory overhead and improving performance. Users requiring strict isolation can add the STRICT ISOLATION characteristic clause to maintain full isolation when needed.

Enhanced SQL Flexibility

Parameter markers (both named :param and unnamed ?) are now supported virtually anywhere a literal value can be used, including in DDL statements and column type definitions. The IDENTIFIER clause has been expanded to work in nearly all contexts where identifiers are permitted, such as column aliases and definitions. Literal string coalescing has also been expanded to any place string literals are allowed.

Infrastructure Updates

The Redshift JDBC driver has been upgraded to version 2.1.0.28, ensuring compatibility with the latest Redshift features and improvements.

All users are automatically upgraded to the latest serverless compute version without requiring code changes, as the environment version maintains a stable client API based on Spark Connect.