web/docs/smart-search.md

Smart search: recall wide, rules in code, a fast judge on top

Every product search goes through one function, findProducts(catalog, request) in src/server/search/service.ts: the catalogue's search box (/ui-api/kitchens/{kitchen}/catalog?q=), the thread's product results (server/thread/home.ts productsCard), the menu planner's ingredient matching (server/agent/menu/match.ts), and the intake's candidates through the core's port (core/catalog/finder.ts, ProductFinder). With the seed catalogue, the crash world's own catalogue and every test, it is the catalogue's own lexical search, unchanged.

How a search runs

1. Read the words (recall.ts understand): terms, dietary claims, allergens to avoid, uses ("for whipping", "kahviin"), product codes, a brand, a pack size. A claim is never dropped: the request's claims are added to the words', and a misspelt claim word ("laktositon") is read as the claim, stricter and never looser, with a note ("read 'laktositon' as lactose-free"). "without soy", "pähkinätön", "utan nötter" become forbidden allergens.
2. Hard rules first, from product data only: claims carried, no forbidden allergen (no allergen data is not safe), the family, an id list. Applied inside every recall source, so a model never sees a product the rules removed and can never bring one back.
3. Recall over all 17,546 products, by cheap means at once, fused by reciprocal rank with a quota per source:
   • lexical: the words in the names (English, Finnish, Swedish; whole or inside a compound), inverted index;
   • typo: words no product name has, corrected to the nearest one that does, or an inflected form to its stem;
   • words: any of the words, rarer words weighing more;
   • kinds: the knowledge map's kinds whose label or aliases carry the words, all their products, lifted when the map profiles them for a use the words name;
   • vectors: the query's embedding (gemini-embedding-2, 768 dimensions) against one vector per product built once from its names, shelf and knowledge-map facts (pnpm search:vectors);
   • code: a product id said outright (then it is a lookup: no embedding, no judge).
4. The judge (judge.ts): one Decisions API call (gemini-decision-1) over the 50 survivors, over a connection pool kept warm (a fresh TLS connection cost 100 to 250 ms; splitting into parallel requests made the tail worse), a request still unanswered at 450 ms sent once more (hedged), all within 800 ms. Each product is exact, variant, standin or no, with a confidence. Late, refused, over quota or switched off: the fused ranking answers, and the result says which path did (path, reason).
5. Code checks the judge: a product the judge calls the thing itself is relabelled a stand-in when it belongs to another family than the words name but can do its job (a vegetable spread for "vegan butter"), or names another animal or fish than the words do (chicken for "turkey mince").
6. Order in code: matches (exact and variant), then stand-ins, never "no". Among matches, the judge's score plus the brand and the pack size the words named (checked from data). onlyStandins when nothing is the kind asked (the thread says "instead"), notSold when nothing in the assortment is, unsure when the top answer is weak.

The intake asks for recall only (no embedding, no judge): the request reader already chose the family, and options must not wait. It adds the family's best text matches that a 100-product category list cuts off, prices the top three first, and looks a family up by recall before refusing words no product name carries.

Fresh produce and allergens

"Unknown is never safe" stays the rule, with one narrow exception (core/catalog/nature.ts): a product the partner files under fresh produce (174xx), that the knowledge map profiles as a vegetable, fruit, herb or potato, whole and fresh, not frozen, with nothing in its names about cutting, peeling, mixing, cooking or coating, contains what it is itself (celeriac is celery) and nothing else. Its evidence reads "by its nature, not declared by the maker" on the product card, in the gate's record and in the signed quote line (allergensByNature). A mandate with allergenEvidence: "declared" keeps the strict rule. Plain lexical search, OpenSearch, alternatives and the crash world stay strict.

Evaluation (14 Nov, 189 requests, fresh)

Times are end to end, including the query embedding. The judge missed its deadline once in 185 searches.

Common practice on the same 189 requests (14 Nov)

The same requests and the same scoring code, run with approaches most teams use, on our catalogue and our index. These are textbook set-ups we built and ran, not other companies' products. They take the buyer's words as typed (no reading of claims or allergens), and the index is the one pnpm search:publish:opensearch creates (on a local Docker OpenSearch 3.3 for this run; BM25 and HNSW are the same algorithms as on the hosted 3.6 service).

• BM25: one multi_match (best fields, any word) with OpenSearch's default BM25 (k1 1.2, b 0.75) over names.fi (the finnish analyzer), names.sv (swedish), names.en (english) and brand (lowercase and ASCII folding); no synonyms, no filters, no rules, no judge; the top 100 are the candidates.
• k-NN: the query embedding (gemini-embedding-2, 768 dimensions) against the same product vectors smart search uses (Lucene HNSW, cosine), top 100, no filters.
• Hybrid (RRF): the BM25 and k-NN top 100 fused by reciprocal rank (k = 60, equal weights): what a standard hybrid search ships, with no rules and no judge.
• + our hard rules: the same lists after the claims and allergens read from the words are applied in code (recall.ts hardFilter), no judge: the rules take "dropped" to 0; the judge (rows above) takes right-first from about 60 % to 97 % and says "we do not sell that".

"Dropped" is a top-3 product that misses a stated claim or carries a forbidden allergen. BM25 times are a local round trip; k-NN and hybrid times include the query embedding's measured time (replayed from the evaluation's cache, about 340 ms).

To reproduce: pnpm search:publish:opensearch against an OpenSearch with the k-NN plugin (OPENSEARCH_URL), then pnpm search:eval --baselines --configs bm25,knn,hybrid_rrf,bm25_rules,hybrid_rrf_rules.

Cost, limits, telemetry

Per search about 0.06 US cents (one query embedding of about 15 tokens at $0.20 per million, about 5,500 judge tokens at $0.10 per million; neither bills output). Smart search's model calls have their own daily count (DOCK_SEARCH_DAILY_CALLS, default 5000) so a busy search box never uses up the calls an order needs; their spend counts in DOCK_LLM_DAILY_USD and search stops calling at that cap's last 10 %. Each call is a telemetry row (search.judge, search.embed) and counts on the conversation's turn.

The vectors on a deployed server

The vectors are derived from partner data and never committed. Locally: pnpm search:vectors writes .data/valio/search-vectors.json.gz (12 MB; about $0.33 for the whole assortment). For a deployed server: pnpm search:publish puts the file in one row of partner_data.search_vectors in the project's PostgreSQL (the schema the read-only Grafana role cannot read), and the server fetches it at boot by hash and caches it on its disk, like the catalogue index and the knowledge map. The server needs GEMINI_API_KEY for the judge and query embeddings; without it, smart search runs recall without vectors or judge.

Evaluation

pnpm search:eval runs the same 181 requests (English, Finnish, Swedish, mixed; 30 held out) through every configuration; --tune leaves out the hold-out, --opus --sample 60 adds the Claude Opus 5.5 reference, --ablation shows what recall is made of. The requests and their expected kinds are in eval/search/cases.ts (kinds from the knowledge map, a sample checked by hand; pnpm tsx eval/search/check-cases.ts validates them). Answers are cached under .data/search-eval/ with the time they took; --fresh measures again.

Product texts in the index (not in the vectors yet)

Since the data-depth index (core/catalog/partner.ts), every IndexedProduct (and the Product it becomes) carries what the maker writes, shortened, in English and Finnish where given: texts.ingredients.{en,fi} (the ingredient list, up to 400 characters, cut at a comma with "…"), texts.storage, texts.usage (usage, else preparation instructions), texts.disposal, texts.about (the first 80 characters of the marketing text), plus mainIngredient.names.{en,fi,sv} with its share and origin, and fish[].latin / fish[].species (FAO code). In the file they are a columnar block (IndexFile.texts, "ingredients.en" ... arrays in row order); pnpm index:build --no-texts leaves them out. The vectors (server/search/docs.ts embeddingText) do not use them: adding them is untested against pnpm search:eval.

Recall on Aiven OpenSearch

Recall can be answered in this process's memory (the inverted index and the vectors above) or by OpenSearch (server/aiven/search/opensearch-recall.ts): one _msearch brings back the lexical, typo and per-word keyword lists (the catalogue index's stemmed, folded and n-gram name fields, a matched word worth more than any text score so "and" ranks products carrying more of the words first) and the nearest vectors (Lucene HNSW, cosine, ef_search 400). The code and kinds sources, the hard filter, the fusion, the quotas and the use lift are the same code for both (recall.ts recallPlan, fuseLists); cheap exact filters (ids, the family's category codes, every claim, no declared or "may contain" forbidden allergen) are pushed into the queries and the hard rules still run in code on what comes back. OpenSearch's own hybrid query is not used: it fuses scores into one list, and the lists are needed to fuse them as memory recall does.

DOCK_SEARCH_RECALL_BACKEND picks the order (recall-router.ts): memory (default) answers from memory and asks OpenSearch only when the vectors are not loaded yet (after a deploy the vectors load after the index when the recall index answers, so the first searches are already semantic) or memory recall fails; opensearch asks OpenSearch first with memory as the fallback; memory-only never asks it. OpenSearch has a deadline (DOCK_SEARCH_RECALL_TIMEOUT_MS, 400) and a breaker (3 failures in a row, 30 s); when both fail, memory recall without vectors answers. Each search's result says which backend answered (stats.recallBackend, recallFallback), /api/health (dataPlane.recall) and the Platform page count them, and each search is one search.done telemetry event (counts, codes and milliseconds only).

The index: pnpm search:publish:opensearch fills dock-recall-v<N+1> and moves the alias dock-recall once the count matches; --rollback moves it back to the previous version, --status shows them, --index <name> fills a development copy without touching the alias, --drop <name> deletes one of its own indexes (never one behind the alias). About 90 s for 17,546 products from a laptop. The catalogue's keyword index (dock-products) is not touched.

Evaluation, 14 Nov, 189 requests, judge50 (the shipped configuration), measured from a laptop in Finland; the OpenSearch run against a development copy of the index on the project's Aiven OpenSearch (3.6):

Per case: the same first product in 185 of 189, and no request right with one backend and wrong with the other; the candidate lists overlap 86 % on average (the keyword lists differ: OpenSearch's analyzers against memory's word index; on a local copy the vector lists alone overlapped 99 % with the exact in-memory ones). One OpenSearch call of 189 missed the 400 ms deadline and memory answered it. The difference in time is the round trip from the laptop (about 80 ms warm, 600 to 900 ms for the first calls on a cold connection and graph: an earlier run that was not warmed up fell back to memory on every search after the breaker opened). So memory stays first and OpenSearch is the fallback; on Aiven Runtime, next to the service, the round trip is shorter but was not measured.