Elasticsearch - Want to sort by field in all indices where that particular field available or not if not then avoid it

Elasticsearch - Want to sort by field in all indices where that particular field available or not if not then avoid it

  1. Blog
  2. 15 November 2026
  3. 3 min read

Stefan Bogdanescu

Founder & Senior Architect

Dynamic Sorting in Elasticsearch When Fields Are Missing Across Indices

You are facing a classic schema-variation problem: you want to sort by a specific field, say status, but that field only exists in a subset of your indices or documents. The error you encountered (illegal_argument_exception) stems from how Elasticsearch handles text fields versus keyword data when performing sorting operations.

The root cause is not the absence of the field in some indices; it is the mapping of the status field to a text type. In Elasticsearch, text fields are optimized for full-text search (tokenization), not for per-document comparisons like sorting or aggregations. Because the engine cannot efficiently compare tokenized strings or booleans stored within an inverted index without loading the actual data into memory, it disables these operations by default. This is why you received the error suggesting to set fielddata=true or use a keyword field.

To resolve this, you must first ensure the mapping is correct. If status is intended for sorting, it should be mapped as a keyword (or boolean) type, not text. You can check your current mappings using:

GET _mapping/_all

If status is currently a text field, you cannot simply enable fielddata globally or per-query without significant memory overhead and potential performance degradation. The recommended approach is to update the mapping to use a keyword or boolean type:

PUT /your-index-name/_mapping
{
  "properties": {
    "status": {
      "type": "boolean" 
    }
  }
}

Once the mapping is corrected to a non-text type, the unmapped_type parameter in your sort clause becomes critical. Your original query included "unmapped_type" => "boolean". This parameter allows Elasticsearch to treat documents missing the field as having an unmapped value of the specified type. However, this behavior has limitations. When sorting by score and a conditional field, you need to define how those documents should rank relative to each other.

If status is absent in an index or document, Elasticsearch will treat it based on the unmapped_type. If you set unmapped_type to boolean, missing values might be treated as null or a specific default, which could skew your sorting order if not handled carefully.

Here is how to structure the query to handle both present and missing fields safely:

"sort" => [
    ["_score" => ["order" => "desc"]],
    ["status" => ["order" => "desc", "unmapped_type" => "boolean"]]
]

This configuration tells Elasticsearch:

  1. Prioritize results by relevance score first.
  2. For secondary sorting, look for status.
  3. If status is missing, assume it is a boolean (likely false) based on the unmapped type.

Trade-offs and Pitfalls

Using unmapped_type effectively treats missing fields as if they had that value. This can introduce bias. If you consider a missing status equivalent to false, documents with an explicit true status will naturally rank higher than those without the field at all, even if the absence of the field shouldn't matter for your business logic.

If your goal is to simply ignore the sort order when the field is missing rather than assigning it a default value, you might consider using a script score or a conditional query to filter out documents where the field is missing before sorting, though this reduces performance due to extra filtering passes.

Another consideration is index consistency. If you are querying multiple indices dynamically (e.g., via _search with an index parameter containing a list), ensure that the mapping update is applied consistently across all relevant indices. Merging mappings or using aliases can help manage this complexity without rewriting data.

For advanced use cases involving dynamic field handling across many indices, tools like LaravelCompany offer insights into managing complex search architectures, but understanding the underlying mapping types remains essential for writing efficient queries.

Our network