Tuning Refresh Interval for Near-Real-Time Search

The refresh interval is the most consequential index setting nobody revisits. It ships at one second, stays at one second, and quietly caps indexing throughput on an index whose users would never notice thirty. This page derives the interval from two measurements — the staleness your product actually tolerates and the cost each candidate imposes — instead of from the default. It sits under index refresh and commit strategies within the data ingestion and synchronization pipelines area.

Prerequisites

  1. An index whose settings you can change and observe for at least an hour under representative load.
  2. Access to _stats/refresh, _stats/merge, and _cat/segments for that index.
  3. A product owner who can answer “how late may this data be?” for each surface that queries the index.
  4. A write workload you can reproduce — a replayed CDC stream or a sampled production trace.

Diagnosis: the default is a guess about your users

One second exists because it is a reasonable compromise for an unknown workload, not because it is right for yours. Two facts make it expensive. First, refresh cost is per shard per interval, whether or not any document was written — a 20-shard index with a trickle of writes performs 20 refreshes every second, most producing near-empty segments. Second, each refresh opens a new searcher, which discards warmed filter caches, so a short interval taxes readers as well as writers.

What makes this hard to notice is that the symptom appears somewhere else. Nobody reports “refresh is too frequent”; they report that indexing is slower than expected, or that query p95 degrades under write load. Correlating those to the refresh setting requires looking at segment creation rate, which almost nobody graphs by default:

# _cat/segments summary on a 20-shard index at refresh_interval=1s, low write rate
shard  segments  docs_per_segment  size
0      148       31                2.1mb        <- 148 segments holding 4,588 docs
1      151       28                1.9mb
...
# The same index at refresh_interval=30s, one hour later:
0      9         512               2.0mb        <- same data, 16x fewer segments

Sixteen times fewer segments means sixteen times less merge work, fewer file handles, faster queries, and a lower steady-state CPU floor — for data that was thirty seconds stale instead of one second stale.

Segment creation and merge work at three refresh intervals Bars showing segments created per hour and the resulting merge megabytes at one second, five seconds and thirty seconds refresh intervals. 1s 3,600 refreshes/hr per shard merge: 8.4 GB rewritten 5s 720/hr 2.6 GB 30s 120/hr 0.9 GB same documents same write rate
Identical data, three intervals. The work removed by a longer interval is work that produced no user-visible benefit.

Solution Steps

1. Derive the tolerance per query surface, not per index

Ask the question per surface: the seller dashboard, the public catalog page, the internal admin search. The index inherits the tightest genuine requirement, and “genuine” means a user would notice and complain — not that it feels nicer.

surface                  requirement          evidence
seller dashboard         < 2s after save      user edits then verifies immediately
public catalog search    < 60s                catalog changes are batched hourly anyway
admin bulk edit review   < 10s                editors work in batches, refresh manually
=> index requirement: 2s, but ONLY for the seller's own writes (see step 4)

2. Measure each candidate under identical load

Run the same replayed workload at each candidate interval and record throughput, segment count, merge bytes, and query p95 together.

#!/usr/bin/env bash
# sweep_refresh.sh — one 15-minute run per candidate interval
for iv in 1s 5s 10s 30s; do
  curl -s -X PUT "localhost:9200/products/_settings" -H 'Content-Type: application/json' \
    -d "{\"index\":{\"refresh_interval\":\"$iv\"}}" > /dev/null
  before=$(curl -s 'localhost:9200/products/_stats/indexing,merge' \
    | jq '[._all.total.indexing.index_total, ._all.total.merges.total_size_in_bytes]')
  ./replay_workload.sh 900            # 15 minutes of representative writes
  after=$(curl -s 'localhost:9200/products/_stats/indexing,merge' \
    | jq '[._all.total.indexing.index_total, ._all.total.merges.total_size_in_bytes]')
  echo "$iv docs=$(jq -n "$after[0]-$before[0]") merged_bytes=$(jq -n "$after[1]-$before[1]")"
done

3. Read the elbow, not the maximum

Throughput improves monotonically as the interval grows, so “biggest interval wins” is not a decision rule. Pick the point where further increases stop paying — usually between 5 and 30 seconds — and stay there.

interval  docs/sec  merged_GB  query_p95_ms   verdict
1s        18,900    8.4        142            baseline
5s        22,100    2.6         96            +17% write, -32% query latency
10s       23,000    1.7         92            +4% over 5s
30s       23,400    0.9         90            +2% over 10s  <- diminishing
=> choose 5s: nearly all the benefit, smallest staleness cost

4. Keep the strict surface strict with a per-request guarantee

The seller dashboard’s two-second requirement does not need to become the index’s interval. Serve that one write with refresh=wait_for and leave the rest of the index on the cheaper setting.

# save_handler.py — strict visibility only where it is actually required
def save_product(es, doc_id: str, source: dict, *, verify_immediately: bool):
    return es.index(
        index="products", id=doc_id, document=source,
        # wait_for blocks THIS request until the next scheduled refresh includes it;
        # it adds no extra refresh work to the shard.
        refresh="wait_for" if verify_immediately else False,
    )

5. Roll out behind a measured comparison

Change one index at a time and hold the previous value for a full traffic cycle so daily seasonality cannot be mistaken for the effect of the change.

# Apply, then annotate the change so dashboards can correlate it later.
curl -s -X PUT 'localhost:9200/products/_settings' -H 'Content-Type: application/json' \
  -d '{"index":{"refresh_interval":"5s"}}'
curl -s -X POST 'http://localhost:3000/api/annotations' -H 'Content-Type: application/json' \
  -d '{"text":"products refresh_interval 1s -> 5s","tags":["search","indexing"]}'
Decision path for choosing a refresh interval A decision tree asking whether any surface needs sub-second visibility, whether the index is bulk loaded, and whether writes are continuous, leading to one second, disabled, or a five to thirty second interval. any surface needing sub-second visibility? yes use wait_for on those writes, not a 1s index no bulk load window in progress? yes disable, restore after no 5–30s from the sweep
Most indices land in the bottom branch. The top branch is a per-request decision, which is why it does not need to change the index setting at all.

Verification

Confirm the refresh rate matches the configured interval by sampling the counter twice:

a=$(curl -s 'localhost:9200/products/_stats/refresh' | jq '._all.total.refresh.total')
sleep 60
b=$(curl -s 'localhost:9200/products/_stats/refresh' | jq '._all.total.refresh.total')
echo "refreshes/min: $((b - a))"
# Expected at 5s across 5 shards with steady writes: ~60  (12 per shard per minute)

Confirm segment growth slowed, which is the mechanism behind the throughput gain:

curl -s 'localhost:9200/_cat/segments/products?h=shard,segment' | wc -l
# Expected: a fraction of the pre-change count once merges catch up (minutes, not hours)

Common Pitfalls

Raising the interval on an index that is search-idle already

An index queried only occasionally stops refreshing automatically after thirty idle seconds, so its measured refresh cost is already near zero and lengthening the interval changes nothing except staleness. Check _stats/refresh before tuning: if the refresh count is far below 3600 / interval per shard per hour, search idle is already doing the work and the setting is not your bottleneck.

Confusing the interval with a freshness guarantee

refresh_interval: 5s bounds how often the shard refreshes, not how long any given document takes to appear end to end. A document must first travel your ingestion pipeline, and a CDC lag of forty seconds makes the refresh interval irrelevant to perceived freshness. Measure end-to-end lag from source commit to searchable, or you will tune the smallest term in the sum.

Changing every index at once

A fleet-wide change makes the effect impossible to attribute: throughput moves, query latency moves, and merge load moves on every index simultaneously, so a regression on one cannot be separated from an improvement on another. Change one index, hold it for a full daily cycle, then move to the next — the discipline costs days and saves the rollback that follows a bad simultaneous change.

Operational notes

A useful sanity check before any of this: confirm the index is actually write-heavy. Refresh cost only matters relative to the work the shard is doing, and on an index taking a handful of writes per minute the entire question is academic — the setting could be one second or one minute with no measurable difference. Spending an afternoon sweeping a low-write index is a common way to produce a beautiful graph that changes nothing.

Once the interval is chosen, the risk is drift rather than the value itself. Refresh settings are edited during incidents — someone disables refresh to push a backfill through and the change survives the incident review — so treat the setting as configuration under version control and reconcile it on every deploy. A reconciliation step that logs “products refresh_interval 30s → 5s (drift corrected)” is far more useful than a runbook step nobody performs.

The second operational habit worth building is re-deriving the tolerance when the product changes. A catalog that gains a seller-facing dashboard has acquired a new visibility requirement that the original sweep never considered, and the honest answer is usually not “shorten the index interval” but “add wait_for to that one write path”. Reviewing the requirement per surface at the same cadence as capacity review — quarterly is plenty — keeps the interval attached to a real user need rather than to whatever seemed safe eighteen months ago.

Finally, remember that this setting interacts with everything else in the write path. Lengthening the interval raises indexing throughput, which means the same hardware absorbs bigger bursts, which changes the operating point you would measure in a throughput sweep. If you tune both, tune refresh first and throughput second; doing it in the other order means re-running the more expensive measurement.

Order of tuning steps for the write path Visibility requirement first, then refresh interval, then bulk throughput sweep, then capacity planning, each feeding the next. 1. visibility per surface 2. interval sweep the elbow 3. throughput size + concurrency 4. capacity plan Reversing steps 2 and 3 forces the expensive measurement to be repeated.
Refresh is upstream of throughput tuning: it changes the cost per document, so measuring throughput first wastes the measurement.