Timeline of ES|QL improvements

ES|QL has evolved since it was first released. New commands, functions, and features have been introduced to cover more use cases. This post looks back at how the language has grown across versions and what each update brought.

What is ES|QL?

ES|QL (Elasticsearch Query Language) is a piped query language built for working with data in Elasticsearch. It allows users to filter, transform, and summarize data using a sequence of commands, which return results in JSON — or in tabular format when specified — and integrates directly with backend workflows through the _query API, or visualizations, dashboards, and alerts in Kibana.

ES|QL was introduced to simplify common data analysis workflows and reduce the need to switch between tools or query languages. The goal was to provide a consistent, readable syntax that works across observability, security, and general-purpose use cases, making it easier to ask complex questions of Elasticsearch data from a single entry point.

For example, a simple ES|QL query might look like this:

FROM kibana_sample_data_logs
| WHERE @timestamp > NOW() - 1d
| KEEP @timestamp, response, message, tags
| SORT response DESC

Example produced with sample web logs dataset.

We have written plenty of articles about ES|QL; I recommend you read them.

The following is a version-by-version summary of ES|QL updates, starting with its initial release.

Pre Launch

Before its first public release in 8.11, ES|QL was developed for more than a year. The team designed and built a dedicated compute and query engine inside Elasticsearch to handle advanced processing close to the data.

Elasticsearch 8.11 (Tech Preview)

ES|QL is introduced as a piped query language in Tech Preview, after months of development to simplify data analysis workflows. This initial release brings functions like LEAST, GREATEST, LEFT, RIGHT, CEIL, and arithmetic with durations and periods. You can compare and sort text fields and TopN to sort rows based on multi-value blocks.

Elasticsearch 8.12

This version focuses on making ES|QL faster and more efficient. The new PROFILE option shows how long each query step takes, helping with debugging and optimization. ES|QL starts supporting GEO_POINT and CARTESIAN_POINT, allowing spatial data to be used in queries. Internal optimizations reduce memory usage and load multiple fields more efficiently. Also, the _source field can now be accessed directly, enabling access to raw document content.

Elasticsearch 8.13

Async query APIs are introduced, allowing long-running queries to be executed in the background and fetched later. New functions like TO_UPPER, TO_LOWER, date_diff, MV_FIRST, and MV_LAST help manipulate strings and multi-value fields. The =~ operator enables case-insensitive comparisons. Cross-cluster enrich queries become possible, meaning you can enrich queries based on remote indices. The STATS command now supports aggregate expressions.

Elasticsearch 8.14 (GA)

ES|QL is now generally available. You can group values inside STATS expressions and sort by expressions, not only by regular fields. Functions like LOCATE (find substring), SIGNUM (return the sign of a given number), and spatial helpers like ST_CONTAINS, ST_DISJOINT, and ST_WITHIN are added. Regex and CASE expressions are optimized. Enrich pipelines support text fields, and serialization of large arrays becomes faster.

Elasticsearch 8.15

You can now cast values explicitly with ::, for example, from string to integer. New aggregation types like top_list help summarize the most frequent values, and the ip_prefix function gets the prefix of an IP address. Arrow format output is added, enabling data operability to analytics tools like pandas or Arrow-compatible engines. Also, the _ignored metadata field is exposed.

Elasticsearch 8.16

Functions like EXP, HYPOT, SPACE, and REVERSE are added. Support for more data types in MIN, MAX, and TOP aggregations, like strings, IPs, and booleans. You can now use MV_PERCENTILE for percentiles on multi-value fields and MV_PSERIES_WEIGHTED_SUM for weighted scoring. Profiling and memory tracking improvements help users better understand their performance and resource usage.

Elasticsearch 8.17

ES|QL starts supporting date_nanos in technical preview, which improves precision for time-series use cases. Scalar functions like BIT_LENGTH and BYTE_LENGTH give you string length in bits/bytes. skip_unavailable now works during query planning, improving fault tolerance in multi-cluster setups. New function metrics allow tracking of which ES|QL functions are most used.

8.18 and 9.0 were released together, but the changelogs are still different.

Elasticsearch 8.18

LOOKUP JOIN becomes a technical preview, letting you join data across indices. For example, matching logs with metadata or user profiles. Full-text scoring is officially supported through _score, and functions like MATCH are improved with new options and compatibility. CATEGORIZE is optimized for large datasets. ST_EXTENT_AGG and ST_ENVELOPE enable spatial summarization, like finding bounding boxes for geo data. date_nanos support is extended across all relevant date functions.

Elasticsearch 9.0

LOOKUP JOIN enters technical preview. KQL (Tech Preview) allows embedding Kibana-style filters into queries.

Full-text match scoring supports disjunctions (e.g., "this OR that"), and _score is now usable in expressions. Advanced telemetry, plan validation, and partial sort fields in TopN pushdowns all contribute to faster, smarter query execution.

Technical enhancements include smarter field cap resolution, telemetry for cross-cluster queries, async metadata for CCS, and support for unmapped fields. Functions like STARTS_WITH, ENDS_WITH, and TO_UPPER are now pushed down to Lucene for better performance.

To learn more about each release in detail, you can check the full timeline we compiled for this article, and if you want to experiment with ES|QL yourself, you can try the ES|QL sandbox.

Ready to try this out on your own? Start a free trial.

Want to get Elastic certified? Find out when the next Elasticsearch Engineer training is running!