Migrating from Elasticsearch to Typesense

Teams migrate away from a JVM search cluster for one reason above all others: the operational burden exceeds what the workload justifies. The migration itself is straightforward — the index is a projection and can be rebuilt — but three things reliably go wrong: features that have no equivalent, query code spread through the application, and a cutover with no way back. This page handles all three. It sits under Meilisearch vs Typesense comparison in search engine selection and architecture.

Prerequisites

  1. A rebuildable index — every document reproducible from a source of truth, as covered in batch versus streaming ingestion.
  2. A confirmed corpus size that fits in memory with headroom, since that is the constraint the target engine imposes.
  3. An inventory of every query the application issues, including the ones in reporting jobs and admin tools.
  4. A set of queries with known-correct results, for comparing the two engines during verification.

Diagnosis: what does not survive the move

Most of the migration is mechanical. A minority of it is not, and finding that minority before starting is the difference between a two-week project and a three-month one.

Nested queries. Elasticsearch’s nested documents preserve the association between fields inside array elements. The target does not have an equivalent, so any query relying on “this variant is both red and large” needs a different data model — typically flattening to one document per variant.

Script scoring. Painless scripts have no counterpart. Anything expressed as a script must be precomputed into a field at ingestion, which is usually possible and occasionally is not.

Aggregations beyond faceting. Terms and numeric facets translate. Nested aggregations, pipeline aggregations and cardinality estimates over large fields generally do not, and reporting queries that use them need a different home — often the source database, which is where they arguably belonged.

Custom analyzers. Language-specific analysis chains, phonetic matching and bespoke token filters are an Elasticsearch strength. The target’s built-in handling is good for common cases and not extensible in the same way.

# Inventory the query shapes actually in use before estimating anything.
grep -rhoE '"(nested|script_score|script|significant_terms|pipeline)"' --include='*.py' --include='*.js' src/ \
  | sort | uniq -c | sort -rn
#   14 "script_score"
#    3 "nested"
# => 17 call sites needing design work, not translation
Which parts of a migration are mechanical and which are design work Documents, mappings and simple queries translate directly, while nested queries, scripts, advanced aggregations and custom analyzers need redesign. mechanical documents · mapping → schema match, filter, sort, facet days of work design work nested → flattened variants scripts → precomputed fields weeks of work the estimate is entirely determined by the size of the right-hand box so inventory it first, before committing to a timeline
The mechanical half is predictable and the design half is not. Counting the right-hand call sites is the first hour of the project.

Solution Steps

1. Put an adapter in front of search before migrating anything

If query construction is spread through request handlers, the migration touches every one of them. Consolidating first — into an interface expressed in your domain’s terms rather than an engine’s DSL — turns the migration into one implementation swap.

# search/port.py — the interface the application uses; no engine concepts
class SearchPort(Protocol):
    def search(self, *, text: str, filters: dict, facets: list[str],
               sort: str | None, page: int, size: int) -> SearchResult: ...

# search/elasticsearch_adapter.py and search/typesense_adapter.py implement it.

Verify: grep for engine-specific vocabulary outside the adapter package. Anything found is a call site the migration would otherwise have missed.

2. Translate the mapping into a schema

{
  "name": "products",
  "fields": [
    {"name": "title",      "type": "string"},
    {"name": "brand",      "type": "string", "facet": true},
    {"name": "categories", "type": "string[]", "facet": true},
    {"name": "price",      "type": "float",  "facet": true},
    {"name": "in_stock",   "type": "bool",   "facet": true},
    {"name": "popularity", "type": "int32"}
  ],
  "default_sorting_field": "popularity"
}

The mapping decisions carry over conceptually: fields you filtered or aggregated on become facetable, fields you only searched stay plain, and fields you never queried should not be indexed at all — the same audit described in schema design and index mapping.

Verify: every field in the schema traces to something the application does. A field present only because the old mapping had it is a field to drop.

3. Load the corpus and measure memory

# Bulk import, then check the actual footprint against available RAM.
curl -s -X POST 'http://localhost:8108/collections/products/documents/import?action=create' \
  -H "X-TYPESENSE-API-KEY: $KEY" --data-binary @products.jsonl | tail -1
curl -s 'http://localhost:8108/metrics.json' -H "X-TYPESENSE-API-KEY: $KEY" \
  | jq '{rss: .system_memory_used_bytes, total: .system_memory_total_bytes}'

Verify: the resident footprint should leave at least 40% headroom for growth and query working memory. If it does not, the target engine is the wrong choice and it is far better to discover that here than after cutover.

4. Run both engines in parallel and compare results

Dual-read is what turns a migration from a leap into a measurement. Serve from the old engine, query both, and log where they disagree.

# shadow.py — serve from the incumbent, compare against the candidate
def search(request):
    primary = es_adapter.search(**request)
    try:
        candidate = ts_adapter.search(**request)
        overlap = len(set(primary.ids[:10]) & set(candidate.ids[:10])) / 10
        metrics.histogram("migration.top10_overlap", overlap)
        if overlap < 0.6:
            log.info("migration.divergence", extra={"query_hash": sha256(request["text"]),
                                                    "primary": primary.ids[:10],
                                                    "candidate": candidate.ids[:10]})
    except Exception:
        metrics.incr("migration.candidate_error")
    return primary

Verify: review the divergence log rather than only the aggregate. Some divergence is expected and fine — the engines rank differently — and what matters is whether the right documents are present in both.

5. Cut over behind a flag, with the old engine still running

SEARCH = ts_adapter if flags.enabled("search.typesense") else es_adapter

A percentage rollout on that flag gives the same properties as an alias swap: instant, reversible, and observable. Keep the old cluster running for a full business cycle, because relevance regressions surface over days rather than minutes.

Verify: the rollback is a flag change, and it should be exercised once deliberately during the rollout so that everyone has seen it work.

Verification

Confirm the corpus is complete:

curl -s 'http://localhost:8108/collections/products' -H "X-TYPESENSE-API-KEY: $KEY" | jq .num_documents
curl -s 'localhost:9200/products/_count' | jq .count
# => the two numbers must match

Confirm known-good queries return the expected documents:

python3 verify_queries.py --engine typesense --expectations expectations.json
# 47/50 expectations met
# 3 failures: all involve nested variant filters — the known design gap

Confirm latency is genuinely better, since that is usually part of the justification:

python3 bench.py --queries production-sample.txt --engines elasticsearch,typesense
# elasticsearch  p50 41 ms  p95 118 ms
# typesense      p50 11 ms  p95  29 ms

Handling the features that do not translate

The design-work items from the diagnosis each have a standard resolution, and knowing them up front turns the estimate from open-ended into countable.

Nested variant queries become one document per variant with the parent’s fields denormalised. The index grows by the average variant count, filtering becomes trivial, and the result list needs grouping to collapse variants back to products. That grouping is a feature the target engine has, so the net change is a larger index and simpler queries.

Script scoring becomes a precomputed numeric field written at ingestion. Anything depending only on document attributes precomputes cleanly; anything depending on the query does not and needs a different approach — usually a small set of discrete boosts chosen by query class rather than a continuous function.

Advanced aggregations move to the source database. Reporting queries that compute distributions over the whole corpus were always an awkward fit for a search engine, and moving them removes both the migration blocker and a source of production load.

Custom analysis is the genuinely hard case. Built-in typo tolerance covers a great deal of what a custom analyzer was doing, but language-specific stemming or phonetic matching may have no equivalent, and that is a legitimate reason not to migrate. Establishing this early — with a test of your actual queries against the target’s default analysis — prevents discovering it after the corpus has been loaded.

Standard resolutions for untranslatable features Nested queries become flattened variants, scripts become precomputed fields, aggregations move to the database, and custom analysis may block the migration. nested → one document per variant scripts → precomputed fields aggregations → the source database custom analysis → may block the move test the fourth box first — it is the only one that can end the project
Three of the four have routine resolutions. The fourth is the one to test in the first week, because it is the only genuine blocker.

Common Pitfalls

Migrating without consolidating query construction first

Without an adapter, every handler that builds a query is a migration site, and the ones in admin tools and reporting jobs are found last — usually in production, after cutover. Consolidating first costs a week and converts an open-ended search-and-replace into a bounded task with a clear finish line.

Assuming relevance will be identical

The two engines rank differently by design, and a migration that expects identical results will treat every difference as a bug. Decide in advance what “acceptable” means — for instance, the known-good document in the top three for 95% of the verification set — and measure against that rather than against the old engine’s exact ordering.

Cutting over without the memory headroom to grow

An index that exactly fits today’s RAM has no room for corpus growth, and the failure when it arrives is abrupt rather than gradual. Size for the projected corpus at the same horizon you would use for shard sizing, and treat a tight fit as a reason to reconsider the target rather than a constraint to accept.

Operational notes

Keep the old cluster running, not merely available. A stopped cluster that must be restarted and caught up is not a rollback path — it is a recovery project. The cost of running both for a fortnight is small next to the option value, and the decommissioning can then happen calmly once the new engine has been through a full traffic cycle including a peak.

Migrate the reporting and admin queries last and deliberately. They are lower traffic and higher complexity, they are frequently the ones using the features that do not translate, and they are also the ones where a temporary regression is tolerable. Doing them first spends the hardest effort on the least important surface.

Migration sequence with the incumbent kept live Consolidate query construction, load and compare in shadow, roll out behind a flag, and decommission only after a full traffic cycle. 1. adapter one call site 2. shadow compare no user impact 3. flag rollout reversible instantly 4. decommission after a full cycle the incumbent stays running through steps 1–3, which is what makes every step reversible
Four steps with a live fallback throughout. The only irreversible action is the last one, and it happens after the evidence is in.