Debugging Scores with the Explain API

“Why is this document ranked third?” is answerable exactly, and most teams never answer it because the explanation output is a deeply nested tree of arithmetic that is genuinely hard to read. This page reduces it to a repeatable procedure: find the decisive contribution, separate text relevance from boosting, and compare two documents to see precisely where they diverge. It sits under custom scoring functions in ranking algorithms and relevance tuning.

Prerequisites

  1. The document id you are asking about and the exact query that produced the ranking, including filters and boosts.
  2. jq or an equivalent, because reading raw explanation JSON without a filter is impractical.
  3. The current ranking configuration, since an explanation is only interpretable against the query that generated it.

Diagnosis: the explanation is a tree, and only one branch matters

An explanation is a recursive structure of value, description and details. The top value is the final score; each level below decomposes it. The mistake is reading it top to bottom — the useful technique is to descend only into the branch with the largest value at each level, which reaches the decisive contribution in three or four steps regardless of tree size.

14.87  sum of:
  12.40  weight(title:runner) ...        <- descend here
   1.92  weight(description:trail)
   0.55  weight(brand:acme)

At the leaf, a BM25 weight decomposes into the three familiar factors, and reading them tells you which one is responsible:

12.40  score(freq=2.0), computed as boost * idf * tf from:
   6.00  boost
   2.94  idf, computed as log(1 + (N - n + 0.5) / (n + 0.5)) from:
       1200  n, number of documents containing term
    412000  N, total number of documents
   0.70  tf, computed as freq / (freq + k1 * (1 - b + b * dl / avgdl)) from:
       2.0  freq
       1.2  k1
      0.75  b
      18.0  dl, length of field
      42.6  avgdl, average field length

That single block answers most questions. A surprisingly low score usually traces to idf — the term is common — or to dl being far above avgdl, meaning length normalisation is penalising the document.

Descending an explanation tree by largest value At each level, following the branch with the largest value reaches the decisive contribution in a few steps. final score 14.87 title 12.40 — descend 1.92 brand 0.55 — ignore boost · idf · tf the answer is one of these three
Following only the largest branch turns an intimidating tree into three or four lookups, and the leaf always names the responsible factor.

Solution Steps

1. Explain a single document against the exact query

curl -s 'localhost:9200/products/_explain/sku-4471' -H 'Content-Type: application/json' -d '{
  "query": {"multi_match": {"query": "trail runner",
            "fields": ["title^6","description"], "type": "best_fields"}}
}' | jq '{matched: .matched, score: .explanation.value}'
# => { "matched": true, "score": 12.4 }

If matched is false, stop: this is a retrieval problem, not a scoring one, and no amount of boost tuning will help.

2. Collapse the tree to the top contributions

curl -s 'localhost:9200/products/_explain/sku-4471' -H 'Content-Type: application/json' -d @query.json \
  | jq -r '[.explanation] | .[0] |
      recurse(.details[]?) | select(.value > 1) |
      "\(.value | . * 100 | round / 100)\t\(.description[0:70])"' | head -12

That one-liner is the workhorse: it flattens the tree and shows only contributions above a threshold, which is almost always enough to identify the cause.

3. Separate text score from function boosts

When a function_score wraps the query, the explanation nests the text score inside the boosting arithmetic. Extracting both separately shows whether the boost or the text match is responsible.

curl -s 'localhost:9200/products/_explain/sku-4471' -H 'Content-Type: application/json' -d @boosted.json \
  | jq '{
      final: .explanation.value,
      text:  (.explanation | recurse(.details[]?) | select(.description | startswith("weight(")) | .value),
      functions: [.explanation | recurse(.details[]?) | select(.description | test("function score")) | .value]
    }'
# => { "final": 31.2, "text": 12.4, "functions": [2.52] }
# 12.4 × 2.52 = 31.2 — the boost is contributing more than the text match.

4. Compare two documents side by side

The most useful form of the question is comparative: why does B outrank A? Explaining both and diffing the flattened output shows exactly where they diverge.

for id in sku-4471 sku-9912; do
  curl -s "localhost:9200/products/_explain/$id" -H 'Content-Type: application/json' -d @query.json \
    | jq -r --arg id "$id" '[.explanation] | .[0] | recurse(.details[]?) |
        select(.value > 0.5) | "\($id)\t\(.value | .*100|round/100)\t\(.description[0:50])"'
done | sort -k3

5. Use the profile API when the question is latency

_explain answers “why this score”; _profile answers “why so slow”. They are different tools and reaching for the wrong one wastes time.

curl -s 'localhost:9200/products/_search' -H 'Content-Type: application/json' -d '{
  "profile": true, "query": {"match": {"title": "trail runner"}}
}' | jq '.profile.shards[0].searches[0].query[] | {type, time: (.time_in_nanos/1e6)}'
# => { "type": "BooleanQuery", "time": 2.84 }

Verification

Confirm the explanation matches the score the search returned, which catches the case where you are explaining a different query than you ran:

SEARCH=$(curl -s 'localhost:9200/products/_search' -H 'Content-Type: application/json' -d @query.json \
  | jq '.hits.hits[] | select(._id=="sku-4471") | ._score')
EXPLAIN=$(curl -s 'localhost:9200/products/_explain/sku-4471' -H 'Content-Type: application/json' -d @query.json \
  | jq '.explanation.value')
echo "$SEARCH vs $EXPLAIN"
# => 12.4 vs 12.4

Confirm a suspected boost is actually firing:

curl -s 'localhost:9200/products/_explain/sku-4471' -H 'Content-Type: application/json' -d @boosted.json \
  | jq -r '.explanation | recurse(.details[]?) | select(.description | test("in_stock")) | .description'
# => "function score, product of: filter(in_stock:true), weight 1.4"
# No output means the filter did not match — usually the real bug.

A worked comparison

The comparative form is where the technique earns its keep, so it is worth seeing end to end. A merchandiser reports that searching “trail runner” ranks a generic shoe above the product actually called Trail Runner. Explaining both and flattening produces:

sku-9912 (the actual "Trail Runner")     sku-4471 (a generic shoe)
  12.40  weight(title:runner) boost 6      6.10  weight(title:runner) boost 6
   2.20  weight(title:trail) boost 6       5.80  weight(description:trail)
   ----                                    9.20  function score: popularity 1.62
  14.60  total                            31.20  total

The diagnosis is immediate and would have been very hard to reach by inspection: the text scores favour the correct product by a wide margin, and a popularity boost of 1.62 on the competitor overturns it. The fix is not a field weight but a cap on the popularity function, and the whole investigation took two commands.

That is the general shape. Text-score differences are usually explicable and usually correct; the surprises are almost always in the multiplicative layer, which is exactly the part that no amount of staring at the query DSL reveals.

Common Pitfalls

Explaining against a different query than production ran

Production queries are usually assembled by code with filters, boosts and user context that a hand-written explanation request omits. The explanation is then correct for a query nobody ran. Log the full query body at a low sampling rate so the exact production query can be replayed, and explain against that.

Reading the explanation of the wrong shard

Term statistics — the idf figures — are per shard by default, so the same document can score differently depending on which shard holds it, and an explanation reflects only its own shard. When comparing two documents that live on different shards, use search_type=dfs_query_then_fetch to compute global statistics, or the comparison is not like for like.

Concluding "the boost is wrong" when the filter never matched

A function whose filter does not match contributes nothing and appears nowhere in the explanation, which reads identically to a function that is absent. Before adjusting a weight, confirm the function appears at all — most “the boost isn’t working” reports are a filter that does not match, not a weight that is too low.

Making explanations available to non-engineers

The people who most often ask “why is this ranked here?” are merchandisers and support staff, and routing every such question through an engineer with a terminal makes the question expensive enough that it stops being asked. A small internal tool changes that economics entirely.

The tool needs very little: a query box, a document identifier, and a rendering of the flattened explanation with the three or four largest contributions labelled in plain language — “text match in title”, “popularity boost”, “in-stock boost”. Nobody outside the search team needs the arithmetic; they need to know which lever moved the result, so they can ask for the right change.

Building it takes a day and it changes the working relationship, because relevance requests arrive as “the popularity boost is too strong for brand queries” rather than as “search is wrong”. That is a considerably more actionable input, and it comes from giving people the same visibility engineers already have.

Raw explanation compared with a labelled internal view The raw tree is engineer-only, while a labelled summary of the largest contributions is usable by merchandising and support. raw explanation tree engineers only labelled top contributions merchandising and support The translation is mechanical, and it converts vague complaints into specific, correct requests.
The same data, two audiences. Exposing the labelled version is one of the highest-leverage day's work available to a search team.

Operational notes

Build the flattening command into a small script and keep it in the repository. The barrier to using _explain is not knowing that it exists but the friction of parsing its output under time pressure, and a explain.sh <doc-id> that prints the top ten contributions removes that barrier permanently.

Explanations are also the best available teaching tool for anyone new to relevance work. Walking through a real ranking with the explanation open turns BM25 from an abstraction into three numbers with visible causes, and it is considerably more effective than any description of the formula — including the one earlier on this page.

Choosing between the explain and profile APIs Explain answers questions about score and ranking, while profile answers questions about latency and which query clause is expensive. _explain "why is this ranked here?" _profile "why is this query slow?" Both are per-shard and both are cheap to run — the mistake is only ever reaching for the wrong one.
Two diagnostics with no overlap. Naming the question first — score or speed — makes the choice automatic.