Elasticsearch Change Field Type: A Comprehensive Guide

Introduction

When working with Elasticsearch, there might be situations where you need to change the field type of an existing index. This article will guide you through the process of changing the field type in Elasticsearch, including the reasons for doing so, the limitations, and the step-by-step instructions.

Reasons for Changing Field Type

Data type mismatch: You might need to change the field type if the data type of the field in the source data has changed, and the new data type is not compatible with the existing field type in Elasticsearch.
Performance optimization: Changing the field type can help optimize the performance of Elasticsearch queries. For example, changing a text field to a keyword field can improve the performance of aggregations and exact match searches.
Improved search relevance: Changing the field type can help improve the search relevance of your Elasticsearch queries. For example, changing a text field to a keyword field can help improve the relevance of exact match searches.

Limitations of Changing Field Type

In-place field type change: Elasticsearch does not support changing the field type of an existing field in-place. To change the field type, you need to create a new index with the updated mapping and reindex the data. If reindexing is not an option, it is also possible to add a sub-field with the new type and update the index in place, but that can potentially increase the size of the index depending on the nature of the field values.
Data loss: Changing the field type can result in data loss if the new field type is not compatible with the existing data. For example, changing a text field to a date field can result in data loss if the existing data cannot be parsed as dates.

Step-by-Step Instructions With Reindexing

1. Create a new index with the updated mapping:

To change the field type, you need to create a new index with the updated mapping. Use the PUT API to create a new index with the desired field type. For example, if you want to change the field type of the title field from text to keyword, you can create a new index with the following mapping:

PUT /new_index
{
  "mappings": {
    "properties": {
      "title": {
        "type": "keyword"
      },
      ...
    }
  }
}

2. Reindex the data:

After creating the new index with the updated mapping, you need to reindex the data from the old index to the new index. Use the _reindex API to copy the data from the old index to the new index:

POST /_reindex
{
  "source": {
    "index": "old_index"
  },
  "dest": {
    "index": "new_index"
  }
}

3. Verify the data in the new index:

Before proceeding, verify that the data in the new index is correct and the field type change has been applied successfully. You can use the _search API to query the new index and check the results:

GET /new_index/_search
{
  "query": {
    "match_all": {}
  }
}

4. Delete the old index (optional):

If you are satisfied with the results in the new index, you can delete the old index to free up resources. Use the DELETE API to delete the old index:

DELETE /old_index

5. Update aliases (optional):

If you are using aliases to reference your indices, update the aliases to point to the new index. Use the _aliases API to update the aliases:

POST /_aliases
{
  "actions": [
    {
      "remove": {
        "index": "old_index",
        "alias": "my_alias"
      }
    },
    {
      "add": {
        "index": "new_index",
        "alias": "my_alias"
      }
    }
  ]
}

Step-by-Step Instructions Without Reindexing

1. Update the mapping of the index:

If reindexing is not an option, you can add a new sub-field in the existing mapping. Use the PUT API to update the index mapping to add a new sub-field with the desired field type. For example, if you want to add a new sub-field of type keyword to the title field of type text, you can update the index mapping as follows:

PUT /index/_mapping
{
  "properties": {
    "title": {
      "type": "text",
      "fields": {
         "keyword": {
            "type": "keyword"
         }
      }
    },
    ...
  }
}

2. Update the data in place:

After updating the mapping, you need to update the existing data to pick up the new mapping. Use the _update_by_query API to refresh the data and populate the new title.keyword sub-field:

POST /index/_update_by_query

When the update task is done, you can update your queries to use the title.keyword field instead of the title field. Please note that this option will probably increase the size of your index depending on the nature of the title values, because the same values will be indexed twice, once as text in the title field and a second time as keyword in the title.keyword field. Always prefer the reindexing option if possible, unless of course, your queries need both fields.

Conclusion

Changing the field type in Elasticsearch requires either creating a new index with the updated mapping and reindexing the data or adding a new sub-field in the index mapping and updating the index in place. This process can help optimize the performance and search relevance of your Elasticsearch queries. However, be aware of the limitations and potential data loss when changing field types. Always verify the data in the new index before deleting the old index or updating aliases.

Ready to try this out on your own? Start a free trial.

Want to get Elastic certified? Find out when the next Elasticsearch Engineer training is running!