Using hybrid search for gopher hunting with Elasticsearch and Go

In the previous parts of this series, it was demonstrated how to use the Elasticsearch Go client for traditional keyword search and vector search. This third part covers hybrid search. We'll share examples of how you can combine both vector search and keyword search using Elasticsearch and the Elasticsearch Go client.

Prerequisites

Just like part one in this series, the following prerequisites are required for this example:

1. Installation of Go version 1.21 or later
2. Create your own Go repo using the recommended structure and package management covered in the Go documentation
3. Creating your own Elasticsearch cluster, populated with a set of rodent-based pages, including for our friendly Gopher, from Wikipedia:

Connecting to Elasticsearch

As a reminder, in our examples, we will make use of the Typed API offered by the Go client. Establishing a secure connection for any query requires configuring the client using either:

1. Cloud ID and API key if making use of Elastic Cloud
2. Cluster URL, username, password and the certificate

Connecting to our cluster located on Elastic Cloud would look like this:

func GetElasticsearchClient() (*elasticsearch.TypedClient, error) {
	var cloudID = os.Getenv("ELASTIC_CLOUD_ID")
	var apiKey = os.Getenv("ELASTIC_API_KEY")

	var es, err = elasticsearch.NewTypedClient(elasticsearch.Config{
		CloudID: cloudID,
		APIKey:  apiKey,
		Logger:  &elastictransport.ColorLogger{os.Stdout, true, true},
	})

	if err != nil {
		return nil, fmt.Errorf("unable to connect: %w", err)
	}

	return es, nil
}

The client connection can then be used for searching, as demonstrated in the subsequent sections.

Manual boosting for hybrid search

When combining any set of search algorithms, the traditional approach has been to manually configure constants to boost each query type. Specifically, a factor is specified for each query, and the combined results set is compared to the expected set to determine the recall of the query. Then we repeat for several sets of factors and pick the one closest to our desired state.

For example, combining a single text search query boosted by a factor of 0.8 with a knn query with a lower factor of 0.2 can be done by specifying the Boost field in both query types, as shown in the below example:

func HybridSearchWithBoost(client *elasticsearch.TypedClient, term string) ([]Rodent, error) {
	var k = 10
	var numCandidates = 10
	var knnBoost float32 = 0.2
	var queryBoost float32 = 0.8

	res, err := client.Search().
		Index("vector-search-rodents").
		Knn(types.KnnSearch{
			Field:         "text_embedding.predicted_value",
			Boost:         &knnBoost,
			K:             &k,
			NumCandidates: &numCandidates,
			QueryVectorBuilder: &types.QueryVectorBuilder{
				TextEmbedding: &types.TextEmbedding{
					ModelId:   "sentence-transformers__msmarco-minilm-l-12-v3",
					ModelText: term,
				},
			}}).
		Query(&types.Query{
			Match: map[string]types.MatchQuery{
				"title": {
					Query: term,
					Boost: &queryBoost,
				},
			},
		}).
		Do(context.Background())

	if err != nil {
		return nil, err
	}

	return getRodents(res.Hits.Hits)
}

The factor specified in the Boost option for each query is added to the document score. By increasing the score of our match query by a larger factor than the knn query, results from the keyword query are more heavily weighted.

The challenge of manual boosting, particularly if you're not a search expert, is that it requires tuning to figure out the factors that will lead to the desired result set. It's simply a case of trying out random values to see what gets you closer to your desired result set.

Reciprocal Rank Fusion in hybrid search & Go client

Reciprocal Rank Fusion, or RRF, was released under technical preview for hybrid search in Elasticsearch 8.9. It aims to reduce the learning curve associated with tuning and reduce the amount of time experimenting with factors to optimize the result set.

With RRF, the document score is recalculated by blending the scores by the below algorithm:

score := 0.0
// q is a query in the set of queries (vector and keyword search)
for _, q := range queries {
    // result(q) is the results 
    if document in result(q) {
        // k is a ranking constant (default 60)
        // rank(result(q), d) is the document's rank within result(q) 
        // range from 1 to the window_size (default 100)
        score +=  1.0 / (k + rank(result(q), d))
    }
}

return score

The advantage of using RRF is that we can make use of the sensible default values within Elasticsearch. The ranking constant k defaults to 60. To provide a tradeoff between the relevancy of returned documents and the query performance when searching over large data sets, the size of the result set for each considered query is limited to the value of window_size, which defaults to 100 as outlined in the documentation.

k and windows_size can also be configured within the Rrf configuration within the Rank method in the Go client, as per the below example:

func HybridSearchWithRRF(client *elasticsearch.TypedClient, term string) ([]Rodent, error) {
	var k = 10
	var numCandidates = 10

	// Minimum required window size for the default result size of 10
	var windowSize int64 = 10
	var rankConstant int64 = 42

	res, err := client.Search().
		Index("vector-search-rodents").
		Knn(types.KnnSearch{
			Field:         "text_embedding.predicted_value",
			K:             &k,
			NumCandidates: &numCandidates,
			QueryVectorBuilder: &types.QueryVectorBuilder{
				TextEmbedding: &types.TextEmbedding{
					ModelId:   "sentence-transformers__msmarco-minilm-l-12-v3",
					ModelText: term,
				},
			}}).
		Query(&types.Query{
			Match: map[string]types.MatchQuery{
				"title": {Query: term},
			},
		}).
		Rank(&types.RankContainer{
			Rrf: &types.RrfRank{
				WindowSize:   &windowSize,
				RankConstant: &rankConstant,
			},
		}).
		Do(context.Background())

	if err != nil {
		return nil, err
	}

	return getRodents(res.Hits.Hits)
}

Conclusion

Here we've discussed how to combine vector and keyword search in Elasticsearch using the Elasticsearch Go client.

Check out the GitHub repo for all the code in this series. If you haven't already, check out part 1 and part 2 for all the code in this series.

Happy gopher hunting!

Resources

1. Elasticsearch Guide
2. Elasticsearch Go client
3. What is vector search? | Elastic
4. Reciprocal Rank Fusion

Related content

ES|QLPython+1

September 9, 2025

Introducing the ES|QL query builder for the Python Elasticsearch Client

Learn how to use the ES|QL query builder, a new Python Elasticsearch client feature that makes it easier to construct ES|QL queries using a familiar Python syntax.

MG

By: Miguel Grinberg

IntegrationsHow To

September 4, 2025

Transforming data interaction: Deploying Elastic’s MCP server on Amazon Bedrock AgentCore Runtime for crafting agentic AI applications

Transform complex database queries into simple conversations by deploying Elastic's search capabilities on Amazon Bedrock AgentCore Runtime platform.

UTSMR+1

By: Udayasimha Theepireddy, Srinivas Pendyala , Matt Ryan and 1 more

Developer ExperienceHow To

September 5, 2025

Running cloud-native Elasticsearch with ECK

Learn to deploy GKE with Terraform and to run Elastic Stack components on Kubernetes with ECK.

ER

By: Eva Ramon

Developer ExperienceHow To

September 1, 2025

Using UBI in Elasticsearch: Creating an app with UBI and search-ui

Learn how to use UBI in Elasticsearch through a practical example. We’ll be creating an application that produces UBI events on search and click results.

EMAD

By: Eduard Martin and Alexander Dávila

ES|QLGenerative AI+1

August 28, 2025

Using ES|QL COMPLETION + an LLM to write a Chuck Norris fact generator in 5 minutes

Discover how to use the ES|QL COMPLETION command to turn your Elasticsearch data into creative output using an LLM in just a few lines of code.

AF

By: Aurélien Foucret