{"id":296,"date":"2026-09-22T17:47:42","date_gmt":"2026-09-22T09:47:42","guid":{"rendered":"https:\/\/wp.qoraapi.com\/grounding-llm-web-search\/"},"modified":"2026-09-22T17:49:45","modified_gmt":"2026-09-22T09:49:45","slug":"grounding-llm-web-search","status":"publish","type":"post","link":"https:\/\/qoraapi.com\/blog\/grounding-llm-web-search\/","title":{"rendered":"Grounding LLM Answers with Web Search"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">A web-grounded answer is not a smarter model call; it is a retrieval pipeline with a model at both ends. Your code decides whether a search is warranted, generates and executes queries, fetches and filters evidence, and only then lets the model write prose over a small, dated evidence set, with citations attached in code rather than by the model. When &#8220;the LLM gave me outdated information&#8221;, the failure is almost always in one of those deterministic stages.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Parametric knowledge is the wrong tool for time-sensitive questions<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Weights encode text observed up to a training cutoff, which makes them excellent at stable knowledge \u2014 language semantics, algorithm design, protocol structure \u2014 and incapable of holding a fact whose truth value changes faster than the model retrains. A model cannot know a value is stale, because staleness is not a property of the text it learned from.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Three failure shapes follow, each needing a different guard.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Confidently stale.<\/strong> The weights hold a value that was correct at training time and is now wrong, stated with the certainty the model applies to arithmetic. Nothing marks it as having moved.<\/li>\n<li><strong>Confidently wrong.<\/strong> The model interpolates a plausible value that was never true: a version number that never existed, a parameter name that sounds right. Fabrication with correct surface form.<\/li>\n<li><strong>Silently missing.<\/strong> The entity or event postdates the cutoff, so there is nothing in memory. The model refuses, hallucinates, or answers a nearby question.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Staleness is a freshness problem: fix it with dated retrieval and a window. Wrongness is a verification problem: fix it with entailment checks against retrieved evidence. Missingness is a routing problem: force retrieval for questions needing post-cutoff knowledge. This is not private-corpus RAG \u2014 chunking and vector-store choices for your own documents are covered in <a href=\"https:\/\/qoraapi.com\/blog\/production-rag-architecture\/\">Production RAG architecture<\/a>. Here the corpus is the open web: uncontrolled, and full of sloppy content.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The grounding loop, step by step<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Eight steps, in order. The interesting engineering is deciding which are model-driven and which must be deterministic, because anything you leave to the model is something you cannot test.<\/p>\n\n\n\n<figure class=\"wp-block-table is-style-stripes\"><table class=\"has-fixed-layout\">\n<thead>\n<tr><th>Step<\/th><th>Driven by<\/th><th>Deterministic obligation<\/th><\/tr>\n<\/thead>\n<tbody>\n<tr><td>1. Route: is a search needed?<\/td><td>Heuristic plus small classifier<\/td><td>Bypass retrieval for stable-knowledge and text-transformation questions<\/td><\/tr>\n<tr><td>2. Generate queries<\/td><td>Small model, structured output<\/td><td>Cap the count, reject empty or duplicate queries, inject the current date<\/td><\/tr>\n<tr><td>3. Execute searches<\/td><td>Code<\/td><td>Timeouts, per-request budget, dedupe by URL, log raw results<\/td><\/tr>\n<tr><td>4. Fetch and extract<\/td><td>Code<\/td><td>Fetch only top-N candidates; strip boilerplate; keep a content hash for caching<\/td><\/tr>\n<tr><td>5. Select evidence<\/td><td>Hard filters in code, ranking model optional<\/td><td>Recency window, domain allowlist, one document per domain, hard drop of the rest<\/td><\/tr>\n<tr><td>6. Compose the answer<\/td><td>Frontier model<\/td><td>Evidence-only prompt, span ids instead of URLs, temperature 0<\/td><\/tr>\n<tr><td>7. Attach citations<\/td><td>Code<\/td><td>Map span ids to URLs from your evidence list; never render a model-authored link<\/td><\/tr>\n<tr><td>8. Verify<\/td><td>Code, optionally a small entailment model<\/td><td>Every cited sentence must be supported by the cited span, or it is dropped or flagged<\/td><\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The model proposes, the code disposes. Evidence selection is a decision; citations are a rendering step.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Query generation: a raw question is a bad search query<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Users write questions for a human who shares their context; search engines need standalone, keyword-bearing queries. Three things break in translation. Conversational deixis (&#8220;my plan&#8221;, &#8220;this error&#8221;) is unresolvable by an index. Question phrasing (&#8220;is it still required?&#8221;) rarely matches document phrasing (&#8220;obligation applies from&#8221;). And most non-trivial questions are multi-hop, needing two or three lookups, each with its own query.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Take a real question: <em>Is the EU AI Act&#8217;s general-purpose AI obligation already in force for models we shipped last year?<\/em> That is three lookups \u2014 the obligation&#8217;s start date, the transition rule for models already on the market, and whether &#8220;shipped last year&#8221; falls inside it.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><code>EU AI Act general purpose AI obligations application date 2026<\/code><\/li>\n<li><code>EU AI Act GPAI transition period models placed on market before August 2025<\/code><\/li>\n<li><code>site:eur-lex.europa.eu AI Act Article 113 entry into force<\/code><\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Two to four queries is the right band. Beyond that you are not improving recall; you are flooding the evidence set with near-duplicates and third-party restatements, pushing the primary source out of the top-N you can afford to fetch. Over-searching also manufactures conflicts you must then resolve.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import json\nfrom dataclasses import dataclass, field\n\nQUERY_SYSTEM = \"\"\"You convert a user question into web search queries.\nRules:\n- Emit 2 to 4 queries. Never emit a single query.\n- Each query must stand alone: no pronouns, no reference to earlier turns.\n- Include at least one query targeting a primary source (regulator, vendor, standards body).\n- If the question is time dependent, put the explicit year or window in the query.\n- If no web lookup is needed, set needs_search to false and queries to [].\nReturn JSON only, matching: {\"needs_search\": bool, \"queries\": [str], \"freshness_days\": int}\nfreshness_days is how old an acceptable source may be for this question.\"\"\"\n\n@dataclass\nclass QueryPlan:\n    needs_search: bool\n    queries: list\n    freshness_days: int\n    notes: list = field(default_factory=list)\n\ndef plan_queries(client, question, today_iso):\n    resp = client.chat.completions.create(\n        model=\"gpt-4o-mini\",\n        temperature=0,\n        response_format={\"type\": \"json_object\"},\n        messages=[\n            {\"role\": \"system\", \"content\": QUERY_SYSTEM},\n            {\"role\": \"user\", \"content\": \"Today is %s. Question: %s\" % (today_iso, question)},\n        ],\n    )\n    raw = json.loads(resp.choices[0].message.content)\n\n    seen, queries = set(), []\n    for q in raw.get(\"queries\", []):\n        q = \" \".join(str(q).split())\n        if len(q) &lt; 6 or q.lower() in seen:\n            continue\n        seen.add(q.lower())\n        queries.append(q)\n    queries = queries[:4]\n\n    notes = []\n    if raw.get(\"needs_search\") and not queries:\n        notes.append(\"planner requested search but produced no usable query\")\n    return QueryPlan(\n        needs_search=bool(raw.get(\"needs_search\")) and len(queries) &gt; 0,\n        queries=queries,\n        freshness_days=max(1, min(int(raw.get(\"freshness_days\") or 30), 3650)),\n        notes=notes,\n    )<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The model writes the queries; the code caps, dedupes and sanitises them. A planner that claims it needs a search but returns nothing usable is a routing bug for your logs, not a silent fallback to an ungrounded call.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Search versus fetch: the two-stage pattern<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A search API returns snippets \u2014 a title, a URL, a date if you are lucky, and 150-300 characters of text. They are cheap and fast, and the worst of both worlds for grounding: stale relative to the live page, truncated mid-sentence, and ranked for click-through, which rewards keyword-stuffed pages with absent or wrong dates. Fetching gives you real content at a price: 200-1500 ms per fetch, tens to hundreds of kilobytes of HTML, and 10-30% of pages unparseable \u2014 JavaScript shells, cookie walls, paywalls, infinite-scroll docs.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So run two stages. Execute the queries, collect every result, deduplicate by normalised URL, keep the best rank per URL. Then fetch only the top three to six candidates by your own ranking, not the search engine&#8217;s, and extract main content. Skip the fetch when the question is a single-hop lookup, the snippet contains the answer as a verifiable token \u2014 a version string, a date, a numeric limit \u2014 and the source is a primary domain.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Source quality: filter evidence before it reaches the model<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Every document in the context window is a vote. Junk evidence does not dilute good evidence so much as hand the model a fluent, confident, wrong thing to summarise. Filtering belongs in code, because a language model judging source reliability from a 1,200-character excerpt is doing the task it is worst at.<\/p>\n\n\n\n<figure class=\"wp-block-table is-style-stripes\"><table class=\"has-fixed-layout\">\n<thead>\n<tr><th>Signal<\/th><th>How to compute it<\/th><th>How to use it<\/th><\/tr>\n<\/thead>\n<tbody>\n<tr><td>Recency<\/td><td>Structured data <code>datePublished<\/code>\/<code>dateModified<\/code>, then meta tags, then a date in the first 400 characters<\/td><td>Hard drop outside a multiple of the question&#8217;s freshness window; unknown age is penalised, not trusted<\/td><\/tr>\n<tr><td>Domain reputation<\/td><td>Allowlist of primary sources (regulator, vendor docs, standards body) plus an explicit denylist<\/td><td>Multiplicative boost; an allowlist match can satisfy a &#8220;primary source present&#8221; gate<\/td><\/tr>\n<tr><td>States a date<\/td><td>Any parseable date, including a visible byline<\/td><td>A page that never says when it was written cannot establish freshness; cap its score<\/td><\/tr>\n<tr><td>Cross-source corroboration<\/td><td>Same claim or same numeric value present on two or more independent domains<\/td><td>Required for high-stakes numeric claims; absent corroboration lowers confidence or triggers abstention<\/td><\/tr>\n<tr><td>Extraction density<\/td><td>Extracted text length and boilerplate ratio after main-content extraction<\/td><td>Drop near-empty extractions; they are usually walls or shells<\/td><\/tr>\n<tr><td>Source independence<\/td><td>Registrable domain of each result<\/td><td>Keep one document per domain, so &#8220;three sources&#8221; means three publishers rather than three syndications of one wire story<\/td><\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">Rank, then drop. A pipeline that keeps twenty results and hopes the model sorts it out is not grounded; it is a summariser with extra steps. Three to six high-quality spans with provenance metadata attached produce better answers and cheaper ones.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Freshness: dates, undated pages, and misdiagnosed bugs<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Freshness is something you compute, not something the model reports. Extract a publication or modification date in a fixed order of preference \u2014 structured data, then meta tags, then the HTTP <code>Last-Modified<\/code> header, then a date in the opening text \u2014 and record which source you used. A date from a copyright footer is not evidence of when the content changed, so treat it as weak.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Undated pages are the common case, not the exception. Treat them as unknown age rather than fresh, and score them middling: with a 30-day freshness window, an undated post about a pricing change is a rumour with a domain name, not a source. Where the requirement is hours rather than days \u2014 status pages, live inventory, release feeds \u2014 skip web search and hit the primary source&#8217;s own API, because the index lags by days.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is where the most common misdiagnosis happens. When a user reports &#8220;the model answered with 2023 numbers&#8221;, the model almost certainly summarised its evidence faithfully. The bug is upstream: a generic query, a ranker that preferred an old high-authority page, a missing recency filter, a cached fetch. Debug the retrieval trace before touching the prompt; <a href=\"https:\/\/qoraapi.com\/blog\/llm-observability\/\">LLM observability<\/a> covers the span structure that makes grounding failures legible.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Citations that actually work<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Asking a model to produce citations produces citations that look right. Two failures hide behind that. The first is a fabricated URL: a plausible link the model never saw. The second is subtler and more common \u2014 the <strong>attribution gap<\/strong>: the URL is real, it was in the evidence, and the sentence it is attached to is not supported by that page. The citation resolves; it just does not support the claim.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The fix is structural: citations are attached in code, from your evidence list, never authored by the model.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Number the selected spans <code>[S1]<\/code>, <code>[S2]<\/code>, and pass them to the model with titles and dates.<\/li>\n<li>Instruct the model to append the span ids it relied on after each factual sentence, and to write no URLs.<\/li>\n<li>Post-process: if the model cites a nonexistent span id, discard the answer and retry with a tighter evidence set. Map surviving ids to URLs from your list.<\/li>\n<li>Verify support: check each cited sentence is entailed by the cited span, using a lexical-overlap floor as a cheap first pass and a small entailment model for the rest.<\/li>\n<li>Drop or mark sentences that fail verification instead of silently keeping them.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Step four is the one teams skip, and the one that turns &#8220;we show sources&#8221; into &#8220;our sources mean something&#8221;.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conflict resolution<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Sources disagree, and the disagreement is usually informative. Three responses are defensible: prefer the more authoritative and more recent source, and say so; surface the disagreement with both values and both dates; or abstain when the conflict is material to the user&#8217;s decision.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Silently picking one is the worst option, and it is the default of every pipeline that concatenates evidence and lets the model write. The model produces one number with full confidence, the user cannot tell whether that was consensus or a coin flip, and the error becomes undetectable. Detect conflict mechanically where you can: extract the numeric or named entity that answers the question from each span and compare. Where extraction is unreliable, ask a small model to classify the evidence as agreeing, conflicting or insufficient, and let code apply the policy.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">When to abstain<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Design the abstention path before the answer path, because the model will not choose it correctly on its own: asked to answer from evidence it almost always answers, and asked to refuse when unsure it refuses too often.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Make abstention a gate in code, evaluated before generation: at least two independent domains, at least one primary or allowlisted source, all evidence inside the freshness window, no unresolved material conflict. When the gate fails, return a structured non-answer \u2014 what was searched, what was found, which condition failed \u2014 plus the raw links so a human can finish the job. A confident wrong number is not a usable outcome, because the user cannot detect it. Track the abstention rate as a first-class metric: a step change is your earliest signal of a retrieval regression.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Cost and latency: worked arithmetic<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Numbers below are a labelled worked example, not a benchmark. Assume $3.00 per million input tokens, $15.00 per million output tokens for the composing model, $0.005 per search API call, and no charge modelled for fetching.<\/p>\n\n\n\n<figure class=\"wp-block-table is-style-stripes\"><table class=\"has-fixed-layout\">\n<thead>\n<tr><th>Stage<\/th><th>Units<\/th><th>Cost<\/th><\/tr>\n<\/thead>\n<tbody>\n<tr><td>Ungrounded answer (1,200 in, 400 out)<\/td><td>1,200 x $3\/M + 400 x $15\/M<\/td><td>$0.0036 + $0.0060 = $0.0096<\/td><\/tr>\n<tr><td>Query planning (600 in, 150 out, small model)<\/td><td>600 x $3\/M + 150 x $15\/M<\/td><td>$0.0018 + $0.0023 = $0.0041<\/td><\/tr>\n<tr><td>Search calls<\/td><td>3 x $0.005<\/td><td>$0.0150<\/td><\/tr>\n<tr><td>Fetched evidence if it reached the model (4 pages x ~2,500 tokens)<\/td><td>10,000 x $3\/M<\/td><td>$0.0300, avoided by filtering in code<\/td><\/tr>\n<tr><td>Composition (3,300 in, 500 out)<\/td><td>3,300 x $3\/M + 500 x $15\/M<\/td><td>$0.0099 + $0.0075 = $0.0174<\/td><\/tr>\n<tr><td>Grounded total<\/td><td>$0.0041 + $0.0150 + $0.0174<\/td><td>$0.0365<\/td><\/tr>\n<\/tbody>\n<\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">So roughly 3.8x the cost of a plain call, with the search API alone contributing 41% of the grounded total. The table also shows the largest available saving: the 10,000 tokens of raw fetched content never need to enter a prompt. Filtering to three spans of about 800 tokens each removes $0.03 of token spend per answer, more than the search calls cost.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Latency is dominated by the network: 300-600 ms for planning, 200-800 ms for three parallel searches, 300-1,500 ms for four parallel fetches, and composition comparable to an ungrounded call. Add one to three seconds end to end, and stream.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Three caching layers pay for themselves. Cache search results keyed by normalised query with a TTL from the freshness class: minutes for status and pricing, a day or more for documentation. Cache fetched and extracted page text keyed by URL plus <code>ETag<\/code> or content hash \u2014 the biggest win, since it eliminates both the fetch and the re-extraction tokens. Cache final answers only when the freshness class tolerates it, because a semantically cached answer to a price question is stale by construction; see <a href=\"https:\/\/qoraapi.com\/blog\/semantic-caching-ai-api\/\">semantic caching for AI APIs<\/a>. Route deliberately: a small model for planning and selection, the frontier model only for composition.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Evaluating groundedness separately from fluency<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Fluency is free and it is not what you are shipping. A grounded answer can be fluent and wrong in a way no readability metric detects, so measure groundedness on its own labelled sample.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Citation precision<\/strong> \u2014 of the citations emitted, what fraction support their sentence. Human-label a few hundred; this is the attribution gap measured directly.<\/li>\n<li><strong>Citation coverage<\/strong> \u2014 what fraction of factual sentences carry a citation at all. Low coverage means the model is writing from memory inside a grounded prompt.<\/li>\n<li><strong>Evidence recall<\/strong> \u2014 did the pipeline retrieve the authoritative source at all? Label the correct URL per question and check it appeared in the selected spans. This separates retrieval failure from composition failure.<\/li>\n<li><strong>Freshness correctness<\/strong> \u2014 did the answer use the current value, from a source inside the required window?<\/li>\n<li><strong>Abstention correctness<\/strong> \u2014 abstained when evidence was insufficient, answered when it was sufficient. Track both directions.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">The labelled set has to be time-varying or it stops testing grounding. Pick questions whose answers genuinely move: a library&#8217;s latest stable release, a regulator&#8217;s filing deadline, a vendor&#8217;s entry-level price. Record the gold answer, the verification date and the gold source URL, then re-verify on a schedule. A static set decays into a memorisation test within months: the model answers from weights, your pipeline looks perfect, and the next question fails. Wire these metrics into the harness described in <a href=\"https:\/\/qoraapi.com\/blog\/llm-eval-harness\/\">building an LLM eval harness<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">A reference implementation<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The pipeline below is the whole loop in miniature: plan, search, fetch-and-extract (stubbed), select with recency and domain filters, compose against numbered spans, then attach and verify citations in code.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import re\nfrom datetime import datetime, timezone\nfrom urllib.parse import urlparse\n\nPRIMARY_DOMAINS = {\"europa.eu\", \"eur-lex.europa.eu\", \"nist.gov\", \"ietf.org\",\n                   \"w3.org\", \"docs.python.org\", \"developer.mozilla.org\"}\nMAX_EVIDENCE = 5\nMIN_TEXT_CHARS = 400\n\nCITE = re.compile(r\"\\[S(\\d+)\\]\")\n\ndef domain_of(url):\n    host = urlparse(url).netloc.lower()\n    return host[4:] if host.startswith(\"www.\") else host\n\ndef domain_trust(url):\n    host = domain_of(url)\n    for d in PRIMARY_DOMAINS:\n        if host == d or host.endswith(\".\" + d):\n            return 1.0\n    return 0.5\n\ndef parse_date(page):\n    for key in (\"dateModified\", \"datePublished\"):\n        if page.get(key):\n            try:\n                return datetime.fromisoformat(str(page[key]).replace(\"Z\", \"+00:00\"))\n            except ValueError:\n                pass\n    m = re.search(r\"\\b(20\\d{2})-(\\d{2})-(\\d{2})\\b\", page.get(\"text\", \"\")[:400])\n    if m:\n        return datetime(int(m[1]), int(m[2]), int(m[3]), tzinfo=timezone.utc)\n    return None\n\ndef score(page, now, freshness_days):\n    published = parse_date(page)\n    if published is None:\n        age_days, recency = None, 0.35      # unknown age is unverifiable, not fresh\n    else:\n        age_days = (now - published).days\n        recency = max(0.0, 1.0 - age_days \/ float(max(freshness_days, 1)))\n    density = min(1.0, len(page.get(\"text\", \"\")) \/ 2000.0)\n    return 0.55 * recency + 0.35 * domain_trust(page[\"url\"]) + 0.10 * density, age_days\n\ndef select_evidence(pages, now, freshness_days):\n    ranked = []\n    for p in pages:\n        if len(p.get(\"text\", \"\")) &lt; MIN_TEXT_CHARS:\n            continue                        # boilerplate, a paywall, or a JS shell\n        s, age = score(p, now, freshness_days)\n        if age is not None and age &gt; freshness_days * 3:\n            continue                        # hard drop, not a soft penalty\n        ranked.append((s, p))\n    ranked.sort(key=lambda t: t[0], reverse=True)\n\n    kept, seen_hosts = [], set()\n    for s, p in ranked:\n        host = domain_of(p[\"url\"])\n        if host in seen_hosts:\n            continue                        # one document per domain keeps sources independent\n        seen_hosts.add(host)\n        kept.append({**p, \"score\": round(s, 3)})\n        if len(kept) == MAX_EVIDENCE:\n            break\n    return kept\n\nCOMPOSE_SYSTEM = \"\"\"Answer using ONLY the numbered evidence spans.\nAfter every factual sentence, append the ids it relies on, e.g. [S1] or [S2][S3].\nWrite no URLs. Cite no span you did not use.\nIf the evidence does not support an answer, reply with exactly: INSUFFICIENT.\"\"\"\n\ndef compose(client, question, evidence):\n    blocks = [\n        \"[S%d] %s | published=%s\\n%s\" % (\n            i, e[\"title\"], e.get(\"published\") or \"unknown\", e[\"text\"][:1200])\n        for i, e in enumerate(evidence, 1)\n    ]\n    resp = client.chat.completions.create(\n        model=\"gpt-4o\",\n        temperature=0,\n        messages=[\n            {\"role\": \"system\", \"content\": COMPOSE_SYSTEM},\n            {\"role\": \"user\", \"content\": \"Evidence:\\n\\n%s\\n\\nQuestion: %s\"\n             % (\"\\n\\n\".join(blocks), question)},\n        ],\n    )\n    return resp.choices[0].message.content.strip()\n\ndef attach_citations(answer, evidence):\n    used = {int(i) for i in CITE.findall(answer)}\n    if any(i &lt; 1 or i &gt; len(evidence) for i in used):\n        return None, \"cited a span that does not exist\"\n    sources = [\n        {\"id\": i, \"url\": e[\"url\"], \"title\": e[\"title\"], \"published\": e.get(\"published\")}\n        for i, e in enumerate(evidence, 1) if i in used\n    ]\n    return {\"answer\": answer, \"sources\": sources}, None   # renderer links [S1] from this list\n\ndef unsupported_sentences(answer, evidence, min_overlap=0.12):\n    problems = []\n    for sentence in re.split(r\"(?&lt;=[.!?])\\s+\", answer):\n        ids = [int(i) for i in CITE.findall(sentence)]\n        if not ids:\n            continue\n        claim = set(re.findall(r\"[a-z0-9]+\", sentence.lower()))\n        best = 0.0\n        for i in ids:\n            span = set(re.findall(r\"[a-z0-9]+\", evidence[i - 1][\"text\"].lower()))\n            best = max(best, len(claim &amp; span) \/ float(max(len(claim), 1)))\n        if best &lt; min_overlap:\n            problems.append(sentence)\n    return problems\n\ndef answer_grounded(client, search_fn, fetch_fn, question, now=None):\n    now = now or datetime.now(timezone.utc)\n    plan = plan_queries(client, question, now.date().isoformat())\n    if not plan.needs_search:\n        return {\"mode\": \"ungrounded\", \"answer\": plain_answer(client, question)}\n\n    pages = []\n    for q in plan.queries:\n        for hit in search_fn(q, limit=5):\n            pages.append({**hit, \"text\": fetch_fn(hit[\"url\"])})\n\n    evidence = select_evidence(pages, now, plan.freshness_days)\n    if len({domain_of(e[\"url\"]) for e in evidence}) &lt; 2:\n        return {\"mode\": \"abstained\", \"reason\": \"fewer than two independent sources\",\n                \"searched\": plan.queries}\n\n    draft = compose(client, question, evidence)\n    if draft == \"INSUFFICIENT\":\n        return {\"mode\": \"abstained\", \"reason\": \"evidence did not support an answer\",\n                \"searched\": plan.queries}\n\n    result, err = attach_citations(draft, evidence)\n    if err:\n        return {\"mode\": \"abstained\", \"reason\": err, \"searched\": plan.queries}\n    result[\"unverified\"] = unsupported_sentences(draft, evidence)\n    result[\"mode\"] = \"grounded\"\n    return result<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">When web grounding is worth the complexity<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Ground it when the truth value of the answer has a shorter shelf life than your deployment cadence: pricing, release versions, availability, regulatory status, deadlines, limits. Ground it when a citation is part of the product requirement \u2014 support replies that must be auditable, compliance-adjacent answers, anything a customer forwards onward.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Do not ground it when the question is about stable concepts: language semantics, algorithm design, historical facts. Those are what the weights are for, and a search round trip adds seconds and failure modes for nothing. Do not ground when the latency budget is under a second, or when the task is generative rather than factual \u2014 summarising a provided document, writing code against a spec. And do not ground against the open web what you already own: if your product answers questions about your own API, index your own docs and retrieve over them.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The pragmatic default is a router, not a policy. Classify the question, ground the time-sensitive and citation-requiring slice, send everything else straight through. Built against a single gateway key rather than each provider separately, <a href=\"https:\/\/qoraapi.com\/\">qoraapi.com<\/a> exposes one OpenAI-compatible endpoint with per-request cost visibility, which makes the arithmetic above verifiable in production rather than theoretical.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Frequently asked questions<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Does giving the model a search tool make its answers grounded?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">No. Tool use gives the model a lever, not a pipeline. A model with a search tool still chooses its own queries, reads unfiltered snippets, judges source credibility itself and writes its own citations \u2014 the decisions you cannot test. Grounding comes from the deterministic stages: recency and domain filtering, one-document-per-domain selection, citations rendered from your evidence list, a verification pass that drops unsupported sentences.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">How many search queries should I generate per question?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Two to four for most questions. One fails on multi-hop questions and on questions whose phrasing does not match document phrasing. Beyond four you are mostly retrieving near-duplicates of the same claim, crowding the top-N candidates you can afford to fetch. Measure evidence recall \u2014 the fraction of questions where the labelled authoritative URL appears in your spans \u2014 and add queries only where it is low.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Why not just use a newer model with a later cutoff?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Because a cutoff moves; it does not disappear. A model trained through next quarter is still wrong about a release published next week, and the failure has the same shape. Freshness is a structural property of a static artefact answering a dynamic question. Newer models help with the confidently-wrong case, since better calibration means more refusals instead of fabrications, but the confidently-stale case is unaffected.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What should I do with pages that have no publication date?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Treat unknown age as a distinct freshness class rather than assuming fresh or discarding outright. Score it below a dated source from the same domain, admit it only when the question&#8217;s freshness requirement is loose, and never let it satisfy a &#8220;recent source present&#8221; gate alone. Where freshness is the whole point, an undated page is an assertion you cannot place in time.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Can the model just tell me which sources it used?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">It will, and the answer will look plausible. A model-generated citation is a generated string, so it can resolve to a real page that does not support the sentence, which is the attribution gap, or to a page you never retrieved. Attach citations in code from evidence you actually fetched, force the model to reference span ids instead of URLs, and verify support after generation.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Web grounding is a retrieval engineering problem wearing a model costume. The model does two things well: turning a question into queries, and writing prose over a small set of curated spans. Everything between those points \u2014 routing, fetching, extraction, recency and domain filtering, evidence selection, citation attachment, support verification, the abstention gate \u2014 should be code you can read, test and trace.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The reason to build it is not accuracy in general; it is that time-sensitive facts are not knowable from weights, and a confidently wrong answer is worse than a visible gap. The reason not to build it is that it costs roughly four times an ungrounded call, adds one to three seconds, and introduces a failure class that only appears when a source changes under you. Decide per question class, not per product.<\/p>\n\n\n\n\n<h3 class=\"wp-block-heading\">Related reading<\/h3>\n\n\n<ul class=\"wp-block-list\"><li><a href=\"https:\/\/qoraapi.com\/blog\/production-rag-architecture\/\">Building Production RAG: Chunking, Hybrid Search, and Re-Ranking<\/a><\/li><li><a href=\"https:\/\/qoraapi.com\/blog\/rag-ingestion-pipeline\/\">Building a RAG Ingestion Pipeline: Crawling, Parsing, and Syncing<\/a><\/li><li><a href=\"https:\/\/qoraapi.com\/blog\/detect-reduce-hallucinations\/\">Detecting and Reducing Hallucinations in Production LLM Apps<\/a><\/li><li><a href=\"https:\/\/qoraapi.com\/blog\/ai-embeddings-rag\/\">AI Embeddings Explained: Vectors, Similarity, and Building Your First RAG<\/a><\/li><\/ul>\n\n","protected":false},"excerpt":{"rendered":"<p>Grounding against the live web fails on freshness, source quality and citations. Build the search loop, filter evidence before the model sees it, attach citations in code, and design an abstain path.<\/p>\n","protected":false},"author":1,"featured_media":295,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[5,6,9,7],"class_list":["post-296","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-api","tag-ai-api","tag-api-gateway","tag-developer-tools","tag-developers"],"_links":{"self":[{"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/posts\/296","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/comments?post=296"}],"version-history":[{"count":1,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/posts\/296\/revisions"}],"predecessor-version":[{"id":314,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/posts\/296\/revisions\/314"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/media\/295"}],"wp:attachment":[{"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/media?parent=296"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/categories?post=296"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/tags?post=296"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}