Elasticsearch One Tip a Day: Using Dynamic Templates To Avoid Rigorous Mappings

In the previous tip we discussed index mappings in Elasticsearch, and how to make managing them an easier task.

But one problem still remains - in real life indexes will consist of more than just a few fields. Often times in fact they will have dozens of them. As if that's not enough, every few fields will require a completely different handling - using a different analyzer or being not analyzed at all for example. And in many systems you will still need to be able to add new fields on the fly.

Creating big mappings, even if they will be applied via an index template, is definitely not fun. Maintaining them later and adding new fields as they are introduced to the system is even more painful, and is a huge friction point. If only there was a way to just tell Elasticsearch to infer a mapping for a field based on its name.

Well, in fact you can do just that. Using dynamic templates you can define field mappings which will be applied to fields based on their type and name.

Consider the following for example:

PUT /my_index
{
    "mappings": {
        "my_type": {
            "dynamic_templates": [
                { "facets": {
                      "match":              "*_facets", 
                      "match_mapping_type": "string",
                      "mapping": {
                          "type":           "string",
                          "index":       	"not_analyzed"
                      }
                }},
                { "text": {
                      "match":              "*", 
                      "match_mapping_type": "string",
                      "mapping": {
                          "type":           "string",
                          "analyzer":       "english"
                      }
                }}
            ]
        }
    }
}

This Mapping will automatically apply an English-language analyzer to all fields of type string, unless their name is suffixed with _facets, in which case that field will be indexed as not_analyzed. This allows to determine on indexing time, as you push the document to Elasticsearch, whether this field needs to be searchable via a stemming analyzer (e.g. an actual text content field), or not analyzed at all to allow it to be used for faceting.

Using this dynamic templates feature combined with index templates which we discussed previously, you can create your mappings once and spend your efforts on writing your actual application. There's no reason for you to worry about the infrastructure so much!