Introducing the ES|QL query builder for the Python Elasticsearch Client

We are excited to announce the new ES|QL query builder module of the Python Elasticsearch client. With it, you can build queries for the ES|QL engine using a familiar Python syntax.

Introduction

ES|QL is a query language specifically designed for data analysis. The new Python query builder makes it easy to construct and run ES|QL queries in your Python application.

The query builder feature is available in current versions of the Python Elasticsearch client, starting from release 8.19. The package is called elasticsearch and can be installed with your favorite Python installation tool. For example, here is how to install it with pip:

pip install elasticsearch

If your application uses a version of the Python Elasticsearch client older than 8.19, then you will need to upgrade it to use the query builder.

Creating ES|QL queries in Python

The following example creates an ES|QL query object using the Python query builder:

from elasticsearch.esql import ESQL

query = (
    ESQL.from_("employees")
    .keep("first_name", "last_name", "height")
    .eval(
        height_feet="height * 3.281",
        height_cm="height * 100",
    )
    .sort("date_hired DESC")
    .limit(3)
)

When printed or converted to a string, this object renders a standard ES|QL query:

>>> print(query)
FROM employees
| KEEP first_name, last_name, height
| EVAL height_feet = height * 3.281, height_cm = height * 100
| SORT date_hired DESC
| LIMIT 3

You can pass the query object directly to the ES|QL endpoint of the Python Elasticsearch client to execute the query:

from elasticsearch import Elasticsearch
client = Elasticsearch(hosts=[os.environ['ELASTICSEARCH_URL']])
response = client.esql.query(query=str(query))

The response object includes a description of all the columns returned and the rows of results:

>>> from pprint import pprint
>>> pprint(response.body)
{'columns': [{'name': 'first_name', 'type': 'text'},
             {'name': 'last_name', 'type': 'text'},
             {'name': 'height', 'type': 'double'},
             {'name': 'height_feet', 'type': 'double'},
             {'name': 'height_cm', 'type': 'double'}],
 'is_partial': False,
 'took': 11,
 'values': [['Adrian', 'Wells', 2.424, 7.953144, 242.4],
            ['Aaron', 'Gonzalez', 1.584, 5.1971, 158.4],
            ['Miranda', 'Kramer', 1.55, 5.08555, 155]]}

The Python ES|QL module includes Python wrappers for all the ES|QL commands, functions, and operators. The next example demonstrates a more advanced query that uses the LENGTH function and a conditional clause:

from elasticsearch.esql import ESQL, functions

query = (
    ESQL.from_("employees")
    .keep("first_name", "last_name", "height")
    .where(functions.length(E("first_name")) < 4)
)

Here is the resulting ES|QL query:

>>> print(query)
FROM employees
| KEEP first_name, last_name, height
| WHERE LENGTH(first_name) < 4

Interested in trying out this feature? Have a look at creating ES|QL queries in the Python Elasticsearch client documentation for additional information.

ES|QL queries with the DSL module

The ES|QL query builder is also integrated with the DSL module of the Elasticsearch client for Python. All document classes include the esql_from() convenience method to create a basic ES|QL query. For example, assuming an Employee document class is defined, a query that returns all the employees (up to a maximum of 1000 results allowed by ES|QL by default) can be created as follows:

query = Employee.esql_from()

This query object can be chained with additional ES|QL commands as needed. To make queries less error-prone, any time a reference to a field needs to be given, the class attribute of the document class can be used instead of a string:

query = (
    Employee.esql_from()
    .where(functions.length(Employee.first_name) < 4)
    .sort(Employee.date_hired.desc())
    .limit(100)
)

The query can be evaluated as above if desired, but document classes provide the esql_execute() method, which runs the query using the DSL module’s managed connection. This method also transforms the raw results into an iterator that returns document instances:

for emp in Employee.esql_execute(query):
    print(f"{emp.first_name} {emp.last_name} is {emp.height:.2f}m tall")

If you want to learn more about the ES|QL query builder integration with the DSL module, take a look at our Elastic documentation.

Conclusion

We hope you will give the new ES|QL query builder a try the next time you need to work with ES|QL in your Python application. Note that this feature is currently released as a technical preview. If you have any questions, feedback or issues, please create an issue in the Python Elasticsearch client repository on GitHub.

Ready to try this out on your own? Start a free trial.

Want to get Elastic certified? Find out when the next Elasticsearch Engineer training is running!