Introducing the ES|QL query builder for the Elasticsearch Ruby Client

We recently released elastic-esql, a Ruby gem published under the Apache 2 license. This gem allows you to build Elastic's ES|QL queries in idiomatic Ruby, which you can then use with the ES|QL query API. ES|QL allows developers to filter, transform, and analyze data stored in Elasticsearch via queries. It uses "pipes" ( | ) to work with the data step by step. The gem uses Ruby functions instead, which you can chain to the original object to build more complex queries:

ESQL:

FROM sample_data | LIMIT 2 | SORT @timestamp DESC

Ruby:

Elastic::ESQL.from('sample_data').limit(2).sort('@timestamp').descending

Installation

The gem can be installed from RubyGems with:

gem install elastic-esql

Or it can be added to a project's Gemfile:

gem 'elastic-esql'

Usage

You can either build a complete query at once or create a query object with a source command like from or row and then chain ES|QL methods to build on it.

query = Elastic::ESQL.from('sample_data')
query.limit(2).sort('@timestamp')

The gem translates the code to ES|QL on the to_s method, so it returns the ES|QL query when it is printed out or is cast as a String:

query = Elastic::ESQL.from('sample_data').limit(2).sort('@timestamp').descending
query.to_s
# => "FROM sample_data | LIMIT 2 | SORT @timestamp DESC"

You can instantiate a query object and mutate its initial state by using the ! equivalents of each function:

query = Elastic::ESQL.from('sample_data')
query.to_s
# => "FROM sample_data"
query.limit!(2).sort!('@timestamp')
query.to_s
# => "FROM sample_data | LIMIT 2 | SORT @timestamp"

The tool provides convenient ways to chain extra steps to an ES|QL function, such as enrich and sort. Once you call enrich on an Elastic::ESQL object, you can chain on and with to it:

esql.enrich!('policy').on('a').with({ name: 'language_name' })

You can also chain desc, asc, nulls_first and nulls_last to your query after using sort:

Elastic::ESQL.from('sample_data').sort('@timestamp').asc.to_s
# => 'FROM sample_data | SORT @timestamp ASC'

Elastic::ESQL.from('sample_data').sort('@timestamp').desc.nulls_first.to_s
# => 'FROM sample_data | SORT @timestamp DESC NULLS FIRST'

It also supports custom Strings, in case you want to write the ES|QL query yourself, or use a feature that hasn't been added to the library yet. custom will join the strings at the end of the query. It will add them as they're sent to the function, without adding any pipe characters. They'll be combined to the rest of the query by a space character.

esql = Elastic::ESQL.from('sample_data')
esql.custom('| MY_VALUE = "test value"').to_s
# => 'FROM sample_data | MY_VALUE = "test value"'

You can also chain custom functions:

esql.custom('| MY_VALUE = "test value"').custom('| ANOTHER, VALUE')
'FROM sample_data | MY_VALUE = "test value" | ANOTHER, VALUE'

Using the ES|QL Query Builder with the Ruby client

You can use the query builder directly with elasticsearch-ruby and the esql.query API by sending the query object:

require 'elasticsearch'
require 'elastic/esql'

client = Elasticsearch::Client.new
index = 'sample_data'

query = Elastic::ESQL.from(index)
                     .sort('@timestamp')
                     .desc
                     .where('event_duration > 5000000')
                     .limit(3)
                     .eval({ duration_ms: 'ROUND(event_duration/1000000.0, 1)' })
client.esql.query(body: { query: query })

You can also use it with the ES|QL Helper from the Elasticsearch Ruby client, find out more:

require 'elasticsearch/helpers/esql_helper'

Elasticsearch::Helpers::ESQLHelper.query(client, query)

As a standalone tool

The gem is designed as a standalone tool to build ES|QL queries in an idiomatic way. It has no runtime dependencies; you can use it with the official Elasticsearch Ruby client, or on its own.

The generated query can be used with the esql.query API in any way an application interacts with the Elasticsearch API (Ruby or not). Once a query is built with elastic-esql, the generated String can be sent to the API as the query parameter in the request body.

I previously wrote about using Elasticsearch with popular Ruby tools. This gem can be used with any of the popular Ruby tools to query Elasticsearch with ES|QL.

Conclusion

This library is in active development, and the final API hasn't been completed yet. It's currently released as a technical preview. If you have any feedback on the current API or general usage, please don't hesitate to open a new issue. Please refer to the README to learn more about the Ruby ES|QL Query Builder.

Ready to try this out on your own? Start a free trial.

Want to get Elastic certified? Find out when the next Elasticsearch Engineer training is running!