Elasticsearch in Javascript the proper way, part I

This is the first article of a series that covers how to use Elasticsearch with JavaScript. In this series, you’ll learn the basics of how to use Elasticsearch in a JavaScript environment and review the most relevant features and best practices to create a search app. By the end, you’ll know everything you need to run Elasticsearch using JavaScript.

In this first part, we will review:

You can check the source code with the examples here.

What is the Elasticsearch Node.js client?

The Elasticsearch Node.js client is a JavaScript library that puts the HTTP REST calls from the Elasticsearch API into JavaScript. This makes it easier to handle and to have helpers that simplify tasks like indexing documents in batches.

Environment

Frontend, backend, or serverless?

To create our search app using the JavaScript client, we need at least two components: an Elasticsearch cluster and a JavaScript runtime to run the client.

The JavaScript client supports all Elasticsearch solutions (Cloud, on-prem, and Serverless), and there are no major differences among them since the client handles all variations internally, so you don’t need to worry about which one to use.

The JavaScript runtime, however, must be run from the server and not directly from the browser.

This is because when calling Elasticsearch from the browser, the user can get sensitive information like the cluster API key, host, or the query itself. Elasticsearch recommends never exposing the cluster directly to the internet and using an intermediate layer that abstracts all this information so that the user can only see parameters. You can read more about this topic here.

We suggest using a schema like this:

In this case, the client only sends the search terms and an authentication key for your server while your server is in total control over the query and communication with Elasticsearch.

Connecting the client

Start by creating an API key following these steps.

Following the previous example, we’ll create a simple Express server, and we’ll connect to it using a client from a Node.JS server.

We’ll initialize the project with NPM and install the Elasticsearch client and Express. The latter is a library to bring up servers in Node.js. Using Express, we can interact with our backend via HTTP.

Let’s initialize the project:

npm init -y

Install dependencies:

npm install @elastic/elasticsearch express split2 dotenv

Let me break it down for you:

@elastic/elasticsearch: It is the official Node.js client
express: It will allow us to spin a lightweight nodejs server to expose Elasticsearch
split2: Splits lines of text into a stream. Useful to process our ndjson files one line at a time
dotenv: Allow us to manage environment variables using a .env file

Create a .env file at the root of the project and add the following lines:

ELASTICSEARCH_ENDPOINT="Your Elasticsearch endpoint"
ELASTICSEARCH_API_KEY="Your Elasticssearch API"

This way, we can import those variables using the dotenv package.

Create a server.js file:

const express = require("express");
const bodyParser = require("body-parser");
const { Client } = require("@elastic/elasticsearch");
 
require("dotenv").config(); //environment variables setup

const ELASTICSEARCH_ENDPOINT = process.env.ELASTICSEARCH_ENDPOINT;
const ELASTICSEARCH_API_KEY = process.env.ELASTICSEARCH_API_KEY;
const PORT = 3000;


const app = express();

app.listen(PORT, () => {
  console.log("Server running on port", PORT);
});
app.use(bodyParser.json());


let esClient = new Client({
  node: ELASTICSEARCH_ENDPOINT,
  auth: { apiKey: ELASTICSEARCH_API_KEY },  
});

app.get("/ping", async (req, res) => {
  try {
    const result = await esClient.info();

    res.status(200).json({
      success: true,
      clusterInfo: result,
    });
  } catch (error) {
    console.error("Error getting Elasticsearch info:", error);

    res.status(500).json({
      success: false,
      clusterInfo: null,
      error: error.message,
    });
  }
});

This code sets up a basic Express.js server that listens on port 3000 and connects to an Elasticsearch cluster using an API key for authentication. It includes a /ping endpoint that when accessed via a GET request, queries the Elasticsearch cluster for basic information using the .info() method of the Elasticsearch client.

If the query is successful, it returns the cluster info in JSON format; otherwise, it returns an error message. The server also uses body-parser middleware to handle JSON request bodies.

Run the file to bring up the server:

node server.js

The answer should look like this:

Server running on port 3000

And now, let’s consult the endpoint /ping to check the status of our Elasticsearch cluster.

curl http://localhost:3000/ping
{
    "success": true,
    "clusterInfo": {
        "name": "instance-0000000000",
        "cluster_name": "61b7e19eec204d59855f5e019acd2689",
        "cluster_uuid": "BIfvfLM0RJWRK_bDCY5ldg",
        "version": {
            "number": "9.0.0",
            "build_flavor": "default",
            "build_type": "docker",
            "build_hash": "112859b85d50de2a7e63f73c8fc70b99eea24291",
            "build_date": "2025-04-08T15:13:46.049795831Z",
            "build_snapshot": false,
            "lucene_version": "10.1.0",
            "minimum_wire_compatibility_version": "8.18.0",
            "minimum_index_compatibility_version": "8.0.0"
        },
        "tagline": "You Know, for Search"
    }
}

Indexing documents

Once connected, we can index documents using mappings like semantic_text for semantic search and text for full-text queries. With these two field types, we can also do hybrid search.

We’ll create a new load.js file to generate the mappings and upload the documents.

Elasticsearch client

We first need to instantiate and authenticate the client:

const { Client } = require("@elastic/elasticsearch");

const ELASTICSEARCH_ENDPOINT = "cluster/project_endpoint";
const ELASTICSEARCH_API_KEY = "apiKey";

const esClient = new Client({
  node: ELASTICSEARCH_ENDPOINT,
  auth: { apiKey: ELASTICSEARCH_API_KEY },
});

Semantic mappings

We’ll create an index with data about a veterinary hospital. We’ll store the information from the owner, the pet, and the details of the visit.

The data on which we want to run full-text search, such as names and descriptions, will be stored as text. The data from categories, like the animal’s species or breed, will be stored as keywords.

Additionally, we’ll copy the values of all fields into a semantic_text field to be able to run semantic search against that information too.

const INDEX_NAME = "vet-visits";

const createMappings = async (indexName, mapping) => {
  try {
    const body = await esClient.indices.create({
      index: indexName,
      body: {
        mappings: mapping,
      },
    });

    console.log("Index created successfully:", body);
  } catch (error) {
    console.error("Error creating mapping:", error);
  }
};

await createMappings(INDEX_NAME, {
  properties: {
    owner_name: {
      type: "text",
      copy_to: "semantic_field",
    },
    pet_name: {
      type: "text",
      copy_to: "semantic_field",
    },
    species: {
      type: "keyword",
      copy_to: "semantic_field",
    },
    breed: {
      type: "keyword",
      copy_to: "semantic_field",
    },
    vaccination_history: {
      type: "keyword",
      copy_to: "semantic_field",
    },
    visit_details: {
      type: "text",
      copy_to: "semantic_field",
    },
    semantic_field: {
      type: "semantic_text",
    },
  },
});

Bulk helper

Another advantage of the client is that we can use the bulk helper to index in batches. The bulk helper allows us to easily handle things like concurrence, retries, and what to do with each document that goes through the function and that succeeds or fails.

An attractive feature of this helper is that you can work with streams. This function allows you to send a file line by line instead of storing the complete file in the memory and sending it to Elasticsearch in one go.

To upload the data to Elasticsearch, create a file called data.ndjson in the project’s root and add the information below (alternatively, you can download the file with the dataset from here):

{"owner_name":"Alice Johnson","pet_name":"Buddy","species":"Dog","breed":"Golden Retriever","vaccination_history":["Rabies","Parvovirus","Distemper"],"visit_details":"Annual check-up and nail trimming. Healthy and active."}
{"owner_name":"Marco Rivera","pet_name":"Milo","species":"Cat","breed":"Siamese","vaccination_history":["Rabies","Feline Leukemia"],"visit_details":"Slight eye irritation, prescribed eye drops."}
{"owner_name":"Sandra Lee","pet_name":"Pickles","species":"Guinea Pig","breed":"Mixed","vaccination_history":[],"visit_details":"Loss of appetite, recommended dietary changes."}
{"owner_name":"Jake Thompson","pet_name":"Luna","species":"Dog","breed":"Labrador Mix","vaccination_history":["Rabies","Bordetella"],"visit_details":"Mild ear infection, cleaning and antibiotics given."}
{"owner_name":"Emily Chen","pet_name":"Ziggy","species":"Cat","breed":"Mixed","vaccination_history":["Rabies","Feline Calicivirus"],"visit_details":"Vaccination update and routine physical."}
{"owner_name":"Tomás Herrera","pet_name":"Rex","species":"Dog","breed":"German Shepherd","vaccination_history":["Rabies","Parvovirus","Leptospirosis"],"visit_details":"Follow-up for previous leg strain, improving well."}
{"owner_name":"Nina Park","pet_name":"Coco","species":"Ferret","breed":"Mixed","vaccination_history":["Rabies"],"visit_details":"Slight weight loss; advised new diet."}
{"owner_name":"Leo Martínez","pet_name":"Simba","species":"Cat","breed":"Maine Coon","vaccination_history":["Rabies","Feline Panleukopenia"],"visit_details":"Dental cleaning. Minor tartar buildup removed."}
{"owner_name":"Rachel Green","pet_name":"Rocky","species":"Dog","breed":"Bulldog Mix","vaccination_history":["Rabies","Parvovirus"],"visit_details":"Skin rash, antihistamines prescribed."}
{"owner_name":"Daniel Kim","pet_name":"Mochi","species":"Rabbit","breed":"Mixed","vaccination_history":[],"visit_details":"Nail trimming and general health check. No issues."}

We use split2 to stream the file lines while the bulk helper sends them to Elasticsearch.

const { createReadStream } = require("fs");
const split = require("split2");
 
const indexData = async (filePath, indexName) => {
  try {
    console.log(`Indexing data from ${filePath} into ${indexName}...`);

    const result = await esClient.helpers.bulk({
      datasource: createReadStream(filePath).pipe(split()),

      onDocument: () => {
        return {
          index: { _index: indexName },
        };
      },
      onDrop(doc) {
        console.error("Error processing document:", doc);
      },
    });

    console.log("Bulk indexing successful elements:", result.items.length);
  } catch (error) {
    console.error("Error indexing data:", error);
    throw error;
  }
};

await indexData("./data.ndjson", INDEX_NAME);

The code above reads a .ndjson file line by line and bulk indexes each JSON object into a specified Elasticsearch index using the helpers.bulk method. It streams the file using createReadStream and split2, sets up indexing metadata for each document, and logs any documents that fail to process. Once complete, it logs the number of successfully indexed items.

Alternatively to the indexData function, you can upload the file directly via UI using Kibana, and use the upload data files UI.

We run the file to upload the documents to our Elasticsearch cluster.

node load.js

Creating mappings for index vet-visits...
Index created successfully: { acknowledged: true, shards_acknowledged: true, index: 'vet-visits' }
Indexing data from ./data.ndjson into vet-visits...
Bulk indexing completed. Total documents: 10, Failed: 0

Searching data

Going back to our server.js file, we’ll create different endpoints to perform lexical, semantic, or hybrid search.

In a nutshell, these types of searches are not mutually exclusive, but will depend on the kind of question you need to answer.

Query type	Use case	Example question
Lexical query	The words or word roots in the question are likely to show up in the index documents. Token similarity between question and documents.	I’m looking for a blue sport t-shirt.
Semantic query	The words in the question are not likely to show up in the documents. Conceptual similarity between question and documents.	I’m looking for clothing for cold weather.
Hybrid search	The question contains lexical and/or semantic components. Token and semantic similarity between question and documents.	I’m looking for an S size dress for a beach wedding.

The lexical parts of the question are likely to be part of titles and descriptions, or category names, while the semantic parts are concepts related to those fields. Blue will probably be a category name or part of a description, and beach wedding is not likely to be, but can be semantically related to linen clothing.

Lexical query (/search/lexic?q=<query_term>)

Lexical search, also called full-text search, means searching based on token similarity; that is, after an analysis, the documents that include the tokens in the search will be returned.

You can check our lexical search hands-on tutorial here.

app.get("/search/lexic", async (req, res) => {
  const { q } = req.query;

  const INDEX_NAME = "vet-visits";

  try {
    const result = await esClient.search({
      index: INDEX_NAME,
      size: 5,
      body: {
        query: {
          multi_match: {
            query: q,
            fields: ["owner_name", "pet_name", "visit_details"],
          },
        },
      },
    });

    res.status(200).json({
      success: true,
      results: result.hits.hits
    });
  } catch (error) {
    console.error("Error performing search:", error);

    res.status(500).json({
      success: false,
      results: null,
      error: error.message,
    });
  }
});

We test with: nail trimming

curl http://localhost:3000/search/lexic?q=nail%20trimming

Answer:

{
    "success": true,
    "results": [
        {
            "_index": "vet-visits",
            "_id": "-RY6RJYBLe2GoFQ6-9n9",
            "_score": 2.7075968,
            "_source": {
                "pet_name": "Mochi",
                "owner_name": "Daniel Kim",
                "species": "Rabbit",
                "visit_details": "Nail trimming and general health check. No issues.",
                "breed": "Mixed",
                "vaccination_history": []
            }
        },
        {
            "_index": "vet-visits",
            "_id": "8BY6RJYBLe2GoFQ6-9n9",
            "_score": 2.560356,
            "_source": {
                "pet_name": "Buddy",
                "owner_name": "Alice Johnson",
                "species": "Dog",
                "visit_details": "Annual check-up and nail trimming. Healthy and active.",
                "breed": "Golden Retriever",
                "vaccination_history": [
                    "Rabies",
                    "Parvovirus",
                    "Distemper"
                ]
            }
        }
    ]
}

Semantic query (/search/semantic?q=<query_term>)

Semantic search, unlike lexical search, finds results that are similar to the meaning of the search terms through vector search.

You can check our semantic search hands-on tutorial here.

app.get("/search/semantic", async (req, res) => {
  const { q } = req.query;

  const INDEX_NAME = "vet-visits";

  try {
    const result = await esClient.search({
      index: INDEX_NAME,
      size: 5,
      body: {
        query: {
          semantic: {
            field: "semantic_field",
            query: q
          },
        },
      },
    });

    res.status(200).json({
      success: true,
      results: result.hits.hits,
    });
  } catch (error) {
    console.error("Error performing search:", error);

    res.status(500).json({
      success: false,
      results: null,
      error: error.message,
    });
  }
});

We test with: Who got a pedicure?

curl http://localhost:3000/search/semantic?q=Who%20got%20a%20pedicure?

Answer:

{
    "success": true,
    "results": [
        {
            "_index": "vet-visits",
            "_id": "-RY6RJYBLe2GoFQ6-9n9",
            "_score": 4.861466,
            "_source": {
                "owner_name": "Daniel Kim",
                "pet_name": "Mochi",
                "species": "Rabbit",
                "breed": "Mixed",
                "vaccination_history": [],
                "visit_details": "Nail trimming and general health check. No issues."
            }
        },
        {
            "_index": "vet-visits",
            "_id": "8BY6RJYBLe2GoFQ6-9n9",
            "_score": 4.7152824,
            "_source": {
                "pet_name": "Buddy",
                "owner_name": "Alice Johnson",
                "species": "Dog",
                "visit_details": "Annual check-up and nail trimming. Healthy and active.",
                "breed": "Golden Retriever",
                "vaccination_history": [
                    "Rabies",
                    "Parvovirus",
                    "Distemper"
                ]
            }
        },
        {
            "_index": "vet-visits",
            "_id": "9RY6RJYBLe2GoFQ6-9n9",
            "_score": 1.6717153,
            "_source": {
                "pet_name": "Rex",
                "owner_name": "Tomás Herrera",
                "species": "Dog",
                "visit_details": "Follow-up for previous leg strain, improving well.",
                "breed": "German Shepherd",
                "vaccination_history": [
                    "Rabies",
                    "Parvovirus",
                    "Leptospirosis"
                ]
            }
        },
        {
            "_index": "vet-visits",
            "_id": "9xY6RJYBLe2GoFQ6-9n9",
            "_score": 1.5600781,
            "_source": {
                "pet_name": "Simba",
                "owner_name": "Leo Martínez",
                "species": "Cat",
                "visit_details": "Dental cleaning. Minor tartar buildup removed.",
                "breed": "Maine Coon",
                "vaccination_history": [
                    "Rabies",
                    "Feline Panleukopenia"
                ]
            }
        },
        {
            "_index": "vet-visits",
            "_id": "-BY6RJYBLe2GoFQ6-9n9",
            "_score": 1.2696637,
            "_source": {
                "pet_name": "Rocky",
                "owner_name": "Rachel Green",
                "species": "Dog",
                "visit_details": "Skin rash, antihistamines prescribed.",
                "breed": "Bulldog Mix",
                "vaccination_history": [
                    "Rabies",
                    "Parvovirus"
                ]
            }
        }
    ]
}

Hybrid query (/search/hybrid?q=<query_term>)

Hybrid search allows us to combine semantic and lexical search, thus getting the best of both worlds: you get the precision of searching by token, together with the meaning proximity of semantic search.

app.get("/search/hybrid", async (req, res) => {
  const { q } = req.query;

  const INDEX_NAME = "vet-visits";

  try {
    const result = await esClient.search({
      index: INDEX_NAME,
      body: {
        retriever: {
          rrf: {
            retrievers: [
              {
                standard: {
                  query: {
                    bool: {
                      must: {
                         multi_match: {
             query: q,
            fields: ["owner_name", "pet_name", "visit_details"],
          },
                      },
                    },
                  },
                },
              },
              {
                standard: {
                  query: {
                    bool: {
                      must: {
                        semantic: {
                          field: "semantic_field",
                          query: q,
                        },
                      },
                    },
                  },
                },
              },
            ],
          },
        },
        size: 5,
      },
    });

    res.status(200).json({
      success: true,
      results: result.hits.hits,
    });
  } catch (error) {
    console.error("Error performing search:", error);

    res.status(500).json({
      success: false,
      results: null,
      error: error.message,
    });
  }
});

We test with “Who got a pedicure or dental treatment?"

curl http://localhost:3000/search/hybrid?q=who%20got%20a%20pedicure%20or%20dental%20treatment

Response:

{
    "success": true,
    "results": [
        {
            "_index": "vet-visits",
            "_id": "9xY6RJYBLe2GoFQ6-9n9",
            "_score": 0.032522473,
            "_source": {
                "pet_name": "Simba",
                "owner_name": "Leo Martínez",
                "species": "Cat",
                "visit_details": "Dental cleaning. Minor tartar buildup removed.",
                "breed": "Maine Coon",
                "vaccination_history": [
                    "Rabies",
                    "Feline Panleukopenia"
                ]
            }
        },
        {
            "_index": "vet-visits",
            "_id": "-RY6RJYBLe2GoFQ6-9n9",
            "_score": 0.016393442,
            "_source": {
                "pet_name": "Mochi",
                "owner_name": "Daniel Kim",
                "species": "Rabbit",
                "visit_details": "Nail trimming and general health check. No issues.",
                "breed": "Mixed",
                "vaccination_history": []
            }
        },
        {
            "_index": "vet-visits",
            "_id": "8BY6RJYBLe2GoFQ6-9n9",
            "_score": 0.015873017,
            "_source": {
                "pet_name": "Buddy",
                "owner_name": "Alice Johnson",
                "species": "Dog",
                "visit_details": "Annual check-up and nail trimming. Healthy and active.",
                "breed": "Golden Retriever",
                "vaccination_history": [
                    "Rabies",
                    "Parvovirus",
                    "Distemper"
                ]
            }
        },
        {
            "_index": "vet-visits",
            "_id": "9RY6RJYBLe2GoFQ6-9n9",
            "_score": 0.015625,
            "_source": {
                "pet_name": "Rex",
                "owner_name": "Tomás Herrera",
                "species": "Dog",
                "visit_details": "Follow-up for previous leg strain, improving well.",
                "breed": "German Shepherd",
                "vaccination_history": [
                    "Rabies",
                    "Parvovirus",
                    "Leptospirosis"
                ]
            }
        },
        {
            "_index": "vet-visits",
            "_id": "8xY6RJYBLe2GoFQ6-9n9",
            "_score": 0.015384615,
            "_source": {
                "pet_name": "Luna",
                "owner_name": "Jake Thompson",
                "species": "Dog",
                "visit_details": "Mild ear infection, cleaning and antibiotics given.",
                "breed": "Labrador Mix",
                "vaccination_history": [
                    "Rabies",
                    "Bordetella"
                ]
            }
        }
    ]
}

Conclusion

In this first part of our series, we explained how to set up our environment and create a server with different search endpoints to query the Elasticsearch documents following the client/server best practices. Stay tuned for part two, in which you’ll learn production best practices and how to run the Elasticsearch Node.js client in Serverless environments.

Ready to try this out on your own? Start a free trial.

Want to get Elastic certified? Find out when the next Elasticsearch Engineer training is running!