{"id":294,"date":"2026-09-22T17:47:31","date_gmt":"2026-09-22T09:47:31","guid":{"rendered":"https:\/\/wp.qoraapi.com\/async-ai-api-jobs-webhooks\/"},"modified":"2026-09-22T17:49:26","modified_gmt":"2026-09-22T09:49:26","slug":"async-ai-api-jobs-webhooks","status":"publish","type":"post","link":"https:\/\/qoraapi.com\/blog\/async-ai-api-jobs-webhooks\/","title":{"rendered":"Async AI APIs: Job Queues, Webhooks, and Long-Running Tasks"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Synchronous request\/response stops working the moment a model call can outlive the infrastructure in front of it. A 60-second load-balancer idle timeout, a 29-second serverless integration limit, or a mobile client that switches networks mid-request will kill a generation that is running fine on the provider side. Those limits are not yours to raise, so decouple submission from completion: the client gets a durable job id immediately, and the result arrives later by poll or webhook.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Why synchronous request\/response breaks down<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Every layer between your user and the model has its own clock, and the smallest one wins. nginx defaults <code>proxy_read_timeout<\/code> to 60 seconds. An AWS ALB idles connections out at 60 seconds. API Gateway enforces a hard 29-second integration timeout you cannot raise. None of them knows the generation is healthy; they terminate on wall-clock time. Serverless sharpens the mismatch: a Lambda can run for 15 minutes, but the front door in front of it caps at 29 seconds.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Mobile and unreliable clients<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A phone that locks its screen, changes networks, or gets backgrounded drops the socket. The work is gone from the client&#8217;s perspective even though you already paid for it. Resuming requires a server-side identity that a synchronous endpoint cannot provide.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Generations that are legitimately slow<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Summarising a 300-page contract, transcribing an hour of audio, generating video, or running an extended-reasoning model through a hard problem takes minutes. No prompt-engineering trick compresses a 90-second generation into a 30-second window. <a href=\"https:\/\/qoraapi.com\/blog\/batch-ai-api-processing\/\">Batch pipelines<\/a> hit this immediately: the larger the batch, the more certain some item blows the budget.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Three delivery patterns compared<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">There are exactly three shapes you can give an asynchronous-capable API, and each is correct under different constraints.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Synchronous with streaming.<\/strong> One connection, tokens arriving as SSE chunks. Low time-to-first-token, unbounded duration, and a dropped connection loses the remainder unless the server supports resumption.<\/li>\n<li><strong>Submit-and-poll.<\/strong> The POST returns <code>202 Accepted<\/code> with a job id; the client polls <code>GET \/jobs\/{id}<\/code> until terminal. Trivial to implement and works from cron and a CLI.<\/li>\n<li><strong>Submit-with-webhook.<\/strong> The server pushes the result to a URL you registered. Lowest latency, largest operational surface.<\/li>\n<\/ul>\n\n\n\n<figure class=\"wp-block-table is-style-stripes\"><table class=\"has-fixed-layout\"><thead><tr><th>Pattern<\/th><th>Client complexity<\/th><th>Delivery latency<\/th><th>Failure behaviour<\/th><th>Works from<\/th><th>Best for<\/th><\/tr><\/thead><tbody><tr><td>Synchronous + streaming<\/td><td>Low<\/td><td>Low to first token; unbounded to completion<\/td><td>Connection drop loses remaining output unless resumable<\/td><td>Any HTTP client, including browsers<\/td><td>Interactive chat, short completions<\/td><\/tr><tr><td>Submit-and-poll<\/td><td>Medium: retry and backoff logic in the client<\/td><td>Bounded by your poll interval<\/td><td>Client can always resume; state lives server-side<\/td><td>Any HTTP client, including CLI and cron<\/td><td>Batch work, internal tools, low volume, no public endpoint<\/td><\/tr><tr><td>Submit-with-webhook<\/td><td>High: public endpoint, HMAC verification, deduplication<\/td><td>Lowest: pushed the moment the job completes<\/td><td>Delivery depends on your endpoint being up; needs retries and a reconciliation sweep<\/td><td>Servers you operate<\/td><td>High-volume, user-facing long jobs<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">These are not mutually exclusive: a chat product streams, a document pipeline submits and webhooks, an internal report polls. Standardising on one shape is the mistake. An <a href=\"https:\/\/qoraapi.com\/blog\/ai-api-gateway-guide\/\">API gateway<\/a> is the natural place to normalise the job contract so clients never learn which provider ran the work.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The job lifecycle state machine<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Model the job explicitly rather than inferring it from a queue message. Six states are enough.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>queued<\/strong> \u2014 accepted and durable, not yet claimed.<\/li>\n<li><strong>running<\/strong> \u2014 a specific worker holds a lease and is accountable for it.<\/li>\n<li><strong>succeeded<\/strong> \u2014 terminal; the result is persisted and retrievable.<\/li>\n<li><strong>failed<\/strong> \u2014 terminal; the error is recorded, no further attempts.<\/li>\n<li><strong>cancelled<\/strong> \u2014 terminal; requested by the client.<\/li>\n<li><strong>expired<\/strong> \u2014 terminal; outlived its deadline or its retention window.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">Persist every transition<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Write each transition to an append-only <code>job_events<\/code> table as well as updating current state on the job row. The row answers &#8220;where is it now&#8221; with one indexed read; the log answers &#8220;what actually happened&#8221; during an incident. Same split as any <a href=\"https:\/\/qoraapi.com\/blog\/llm-observability\/\">observability pipeline<\/a>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Why &#8220;running&#8221; must have a lease<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A <code>running<\/code> state with no owner is a latent bug. If a worker crashes mid-call, a naive implementation leaves the job running forever: no retry, no alert, nothing for a reaper to match. Define <code>running<\/code> as &#8220;worker W holds a lease expiring at T&#8221;, renewed by a heartbeat, and a reaper reclaims any row where <code>lease_expires_at &lt; now()<\/code>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Lease length is a real trade-off. Too short and a slow-but-healthy job is reclaimed and run twice; too long and a crashed worker blocks it for the full duration. A 60-second lease with a 15-second heartbeat tolerates three missed heartbeats, absorbing a GC pause without sluggish recovery.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Queue design<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Choosing a broker<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">You probably do not need Kafka. For tens of thousands of jobs per day, Postgres with <code>SELECT ... FOR UPDATE SKIP LOCKED<\/code> is a complete transactional queue and removes a piece of infrastructure. It also lets the job row and the queue entry commit in the same transaction, which is the property you want.<\/p>\n\n\n\n<figure class=\"wp-block-table is-style-stripes\"><table class=\"has-fixed-layout\"><thead><tr><th>Broker<\/th><th>Delivery semantics<\/th><th>Visibility timeout<\/th><th>Best fit<\/th><\/tr><\/thead><tbody><tr><td>Postgres SKIP LOCKED<\/td><td>At-least-once<\/td><td>A column you manage yourself<\/td><td>Under 1M jobs\/day; transactional enqueue, no extra infrastructure<\/td><\/tr><tr><td>Redis Streams<\/td><td>At-least-once<\/td><td>Pending entries list plus XCLAIM<\/td><td>High throughput, low latency, consumer groups<\/td><\/tr><tr><td>SQS<\/td><td>At-least-once; FIFO adds exactly-once processing within the queue<\/td><td>Native, extendable up to 12 hours<\/td><td>Managed, AWS-native, native dead-letter redrive<\/td><\/tr><tr><td>RabbitMQ<\/td><td>At-least-once<\/td><td>Native, per-consumer<\/td><td>Routing rules, priorities, per-message TTL<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">At-least-once is what you get<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">End-to-end exactly-once delivery does not exist. What you can build is exactly-once <em>effects<\/em>: at-least-once delivery plus idempotent consumers. SQS FIFO offers exactly-once processing within a queue via a five-minute dedup window, but that guarantee stops at the queue boundary.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Visibility timeout<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A visibility timeout is the window during which a claimed message is hidden from other consumers. If it expires before you finish, the message is redelivered and a second worker starts the same job. It must exceed your p99 processing time, or the worker must extend it. SQS exposes <code>ChangeMessageVisibility<\/code>; on Postgres, the lease heartbeat.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Dead-letter queues<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">After N failed receives, move the message to a DLQ instead of retrying forever. A poison message that fails deterministically \u2014 malformed payload, retired model, empty account \u2014 otherwise consumes worker capacity indefinitely. Alarm on DLQ depth: an empty DLQ is healthy, one growing for an hour is an incident.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Never call the provider inside the transaction<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">This is the most common design error in job systems. The pattern looks reasonable: begin a transaction, insert the job row, call the model, update to succeeded, commit. It is wrong for three reasons.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>It holds a transaction open for the entire external call. Connection pools are small, and a handful of slow generations exhausts them, stalling unrelated parts of the application.<\/li>\n<li>If the call succeeds but the transaction rolls back, you have paid for a generation and thrown the result away.<\/li>\n<li>If the transaction commits but the call fails, you have a job row asserting success with no result behind it.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Separate them. Commit the job row and an outbox event in one short transaction, then have a dispatcher publish after that commit. The outbox pattern makes persist-and-enqueue atomic, at the cost of a small bounded delay.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Webhook consumer correctness<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">A webhook endpoint is an unauthenticated public write path into your system. Treat it with the suspicion you would apply to any other one.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Verify the signature over the raw body<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Compute the HMAC over the exact bytes you received, before JSON parsing or re-serialisation. Most verification bugs come from hashing a re-serialised object whose key order or whitespace no longer matches. Read the raw buffer, verify, then parse, and compare in constant time.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Reject replays with a timestamp window<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Sign the timestamp with the payload and reject anything outside a tolerance window \u2014 five minutes is practical. Without a window, a captured request is valid forever, and the window must absorb clock skew and retry delay.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Deduplicate by event id<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Every provider sends an event identifier. Insert it into a table with a unique constraint and treat a conflict as already processed. This is the only reliable defence against duplicate delivery: retries, network duplication and your own redeploys all produce them.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Return 2xx fast, then process<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Do the minimum work in the handler: verify, deduplicate, enqueue, return <code>200<\/code>. Everything else runs in a worker. If you process synchronously and exceed the provider&#8217;s delivery timeout \u2014 commonly 5 to 10 seconds \u2014 the provider marks delivery failed and retries, so you do the work twice while also being slow.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What happens when your endpoint is down<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Providers retry with exponential backoff for hours to a few days, then give up \u2014 a few fast retries, widening intervals, then silence. Your endpoint must be idempotent, because it will see the same event at one second and again at six hours. Webhooks also cannot be your only source of truth.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Treat webhooks as the fast path and a periodic reconciliation sweep as the slow path. The sweep queries the provider for anything running long and settles it locally. The webhook makes the system fast; the sweep makes it correct.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Idempotency for long jobs<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Long jobs are expensive, which makes duplicate execution expensive. Two layers of protection are needed.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Idempotency keys on submission<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Let the client supply an idempotency key on the POST, stored under a unique constraint scoped to the tenant, so a retried submission returns the original job id instead of creating a second job. The POST will be retried whether you plan for it or not: by the client&#8217;s HTTP library, by a load balancer, or by a double-click.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Bind the key to a hash of the request body. If the same key arrives with a different body, return <code>409 Conflict<\/code> rather than silently returning the first job&#8217;s result. Otherwise a client reusing a constant key gets plausible-looking answers to questions it never asked.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Provider idempotency support varies<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Some providers accept an <code>Idempotency-Key<\/code> header on some endpoints. Others document none, or scope it to a short window or a subset of routes. You cannot depend on the provider to deduplicate your retries, so design as though every retry may produce a second generation and a second charge.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Make your own side effects safe<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Every effect your worker performs needs a key that makes repeating it harmless.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Result writes: <code>INSERT ... ON CONFLICT (job_id) DO NOTHING<\/code>.<\/li>\n<li>Billing: a ledger entry keyed by <code>(job_id, 'completion')<\/code> under a unique constraint, so a redelivered completion cannot double-charge. Same discipline as <a href=\"https:\/\/qoraapi.com\/blog\/ai-usage-metering-billing\/\">usage metering and billing<\/a>.<\/li>\n<li>Notifications: deduplicate on <code>(job_id, channel)<\/code> before sending.<\/li>\n<li>Downstream calls: propagate the job id as the downstream idempotency key.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Where an effect cannot be made idempotent, record the provider&#8217;s request identifier as soon as you receive it and, on retry, query that job&#8217;s status instead of resubmitting. That converts a duplicate generation into a cheap status lookup.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Polling done right<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Polling is not the inferior pattern. It is correct when you have no public endpoint, low volume, or a need for a single authoritative state store &mdash; it just has to be done with backoff.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Exponential backoff with jitter<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Fixed-interval polling wastes requests early and is too slow late. Exponential backoff with full jitter \u2014 sleeping a uniform random amount between zero and <code>min(cap, base * 2^attempt)<\/code> \u2014 spreads load and cuts request volume by roughly an order of magnitude. Without jitter, clients that started together stay synchronised and reproduce identical thundering-herd bursts.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Honour Retry-After<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">When a status endpoint returns <code>429<\/code> or <code>503<\/code> with a <code>Retry-After<\/code> header, that value overrides your backoff. Ignoring it is how well-behaved clients get throttled into uselessness, and it causes the <a href=\"https:\/\/qoraapi.com\/blog\/ai-api-rate-limits-429-errors\/\">429 storms<\/a> that look like provider outages but are self-inflicted.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Long polling<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Long polling holds the connection open until the result is ready or a server-side timeout of 30 to 60 seconds elapses, collapsing many status requests into one. Most LLM providers do not offer it, so in practice it applies at your own gateway.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">The cost of polling at scale<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Assume 50,000 jobs per day with an average completion time of 4 minutes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Naive polling every 2 seconds: 240 divided by 2 gives 120 polls per job, so 6,000,000 status requests per day. Spread over 86,400 seconds that is about 69 requests per second \u2014 and since submissions cluster in business hours, the peak is several times that.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Apply backoff starting at 1 second, doubling, capped at 30 seconds. Cumulative poll times are 1, 3, 7, 15, 31, 61, 91, 121, 151, 181, 211 and 241 seconds \u2014 12 polls to cover a 240-second job, roughly a 10x reduction. The same jobs now generate 600,000 requests per day, about 7 per second.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The architectural point is where those requests land. Clients should poll <em>your<\/em> job table, never the provider directly. Your status endpoint is one indexed primary-key read; a provider status call consumes rate-limit budget and may cost money. Keep one background reconciler as the only component that talks to the provider.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Progress reporting and cancellation<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Progress reporting should be coarse and honest. Expose the state, an attempt count, and a percentage only if the provider actually reports one. A synthesised percentage that creeps toward 90 percent and then stalls is worse than a plain &#8220;running&#8221; label.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Cancellation support varies by provider and endpoint. Some accept a cancel call that terminates a running job. Some only cancel jobs that have not started. Some have none at all, and the generation completes and bills regardless. Establish which before putting a cancel button in a UI.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">When a job cannot be cancelled, fall back to compensating actions: mark it cancelled locally so the result is discarded on arrival, stop downstream work, and refund if you billed at submission. Make sure the failover layer knows too, or a cancelled job gets failed over and billed twice.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">A worker that claims a job with a lease<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">This worker uses Postgres as the queue. It claims one job atomically, records the lease owner, calls the model outside any transaction, and writes the result only if it still holds the lease &mdash; which stops a reclaimed job from being written twice.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import os\nimport time\nimport uuid\nimport psycopg\nfrom openai import OpenAI\n\nLEASE_SECONDS = 60\nMAX_ATTEMPTS = 3\n\nclient = OpenAI(\n    api_key=os.environ[\"QORA_API_KEY\"],\n    base_url=\"https:\/\/api.qoraapi.com\/v1\",\n)\n\nCLAIM_SQL = \"\"\"\nUPDATE jobs\n   SET state = 'running',\n       attempt = attempt + 1,\n       lease_owner = %(worker)s,\n       lease_expires_at = now() + make_interval(secs => %(lease)s),\n       started_at = COALESCE(started_at, now())\n WHERE id = (\n     SELECT id FROM jobs\n      WHERE state = 'queued'\n         OR (state = 'running' AND lease_expires_at &lt; now())\n      ORDER BY created_at\n      FOR UPDATE SKIP LOCKED\n      LIMIT 1)\nRETURNING id, attempt, payload;\n\"\"\"\n\nFINISH_SQL = \"\"\"\nUPDATE jobs\n   SET state = %(state)s,\n       result = %(result)s,\n       error = %(error)s,\n       lease_owner = NULL,\n       lease_expires_at = NULL,\n       finished_at = now()\n WHERE id = %(id)s\n   AND lease_owner = %(worker)s\nRETURNING id;\n\"\"\"\n\ndef claim(conn, worker):\n    with conn.cursor() as cur:\n        cur.execute(CLAIM_SQL, {\"worker\": worker, \"lease\": LEASE_SECONDS})\n        return cur.fetchone()\n\ndef finish(conn, worker, job_id, state, result=None, error=None):\n    with conn.cursor() as cur:\n        cur.execute(FINISH_SQL, {\n            \"id\": job_id, \"worker\": worker,\n            \"state\": state, \"result\": result, \"error\": error,\n        })\n        # No row returned means the lease was reclaimed; discard our result.\n        return cur.fetchone() is not None\n\ndef run(worker):\n    with psycopg.connect(os.environ[\"DATABASE_URL\"]) as conn:\n        conn.autocommit = True\n        while True:\n            job = claim(conn, worker)\n            if job is None:\n                time.sleep(1)\n                continue\n            job_id, attempt, payload = job\n            try:\n                resp = client.chat.completions.create(\n                    model=payload[\"model\"],\n                    messages=payload[\"messages\"],\n                    timeout=600.0,\n                )\n                text = resp.choices[0].message.content\n                finish(conn, worker, job_id, \"succeeded\", result=text)\n            except Exception as exc:\n                state = \"failed\" if attempt &gt;= MAX_ATTEMPTS else \"queued\"\n                finish(conn, worker, job_id, state, error=str(exc))\n\nif __name__ == \"__main__\":\n    run(f\"worker-{uuid.uuid4()}\")\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The same lease concept as SQL, which is also what a reaper runs on a timer:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>-- Reclaim jobs whose worker stopped heartbeating, while attempts remain.\nUPDATE jobs\n   SET state = 'queued',\n       lease_owner = NULL,\n       lease_expires_at = NULL\n WHERE state = 'running'\n   AND lease_expires_at &lt; now()\n   AND attempt &lt; 3;\n\n-- Give up on jobs that exhausted their attempts.\nUPDATE jobs\n   SET state = 'failed',\n       error = COALESCE(error, 'lease expired after max attempts')\n WHERE state = 'running'\n   AND lease_expires_at &lt; now()\n   AND attempt &gt;= 3;\n<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Webhook verification in practice<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The handler below is deliberately boring: verify, deduplicate, enqueue, acknowledge. The unusual detail is that it works on the raw request buffer, which is what makes the signature check correct.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import crypto from \"node:crypto\";\nimport type { Request, Response } from \"express\";\nimport { pool } from \".\/db\";\nimport { enqueue } from \".\/queue\";\n\nconst TOLERANCE_SECONDS = 300;\n\nfunction verify(rawBody: Buffer, header: string, secret: string): boolean {\n  const parts = Object.fromEntries(\n    header.split(\",\").map((kv) => kv.split(\"=\") as [string, string]),\n  );\n\n  const timestamp = Number(parts.t);\n  if (!Number.isFinite(timestamp)) return false;\n  if (Math.abs(Date.now() \/ 1000 - timestamp) &gt; TOLERANCE_SECONDS) return false;\n\n  const expected = crypto\n    .createHmac(\"sha256\", secret)\n    .update(`${parts.t}.`)\n    .update(rawBody)\n    .digest(\"hex\");\n\n  const a = Buffer.from(expected);\n  const b = Buffer.from(parts.v1 ?? \"\");\n  return a.length === b.length && crypto.timingSafeEqual(a, b);\n}\n\n\/\/ Mount with express.raw({ type: \"application\/json\" }) so req.body is a Buffer.\nexport async function webhookHandler(req: Request, res: Response) {\n  const raw = req.body as Buffer;\n  const header = req.header(\"x-provider-signature\") ?? \"\";\n\n  if (!verify(raw, header, process.env.WEBHOOK_SECRET ?? \"\")) {\n    res.status(401).end();\n    return;\n  }\n\n  const event = JSON.parse(raw.toString(\"utf8\"));\n\n  const inserted = await pool.query(\n    `INSERT INTO webhook_events (event_id, received_at)\n     VALUES ($1, now()) ON CONFLICT (event_id) DO NOTHING\n     RETURNING event_id`,\n    [event.id],\n  );\n\n  \/\/ Duplicate delivery: acknowledge without reprocessing.\n  if (inserted.rowCount === 0) {\n    res.status(200).end();\n    return;\n  }\n\n  await enqueue(\"job-results\", { eventId: event.id, payload: event.data });\n  res.status(200).end();\n}\n<\/code><\/pre>\n\n\n\n<h2 class=\"wp-block-heading\">Operational concerns<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">The stuck-job reaper<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Run the reaper every 30 to 60 seconds and make it idempotent, so a double run is harmless. Alert when one pass reclaims more than a small threshold: a burst means workers are crashing in bulk, an infrastructure problem rather than bad luck.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Alert on queue age, not just depth<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Queue depth alone is misleading. Ten thousand pending jobs is comfortable at 50 milliseconds each and catastrophic at five minutes each. Alert on the age of the oldest unclaimed message and the p95 time from <code>queued<\/code> to <code>running<\/code>.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Per-tenant fairness<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">One tenant submitting 100,000 jobs must not starve everyone else. The cheapest control is a per-tenant concurrency cap enforced at claim time, so no tenant holds more than N worker slots. Weighted fair queueing is the fuller answer: order candidates by how far each tenant sits below its fair share.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Backpressure when the provider is slow<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">When the provider returns 429s or latency climbs, the instinct is to retry harder. That is backwards. Slow the consumers, honour <code>Retry-After<\/code>, and trip a circuit breaker after consecutive failures. Bound the queue and reject new submissions with <code>429<\/code> once full: refusing a job immediately beats accepting one you cannot finish.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">When webhooks are worth it<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">My default is submit-and-poll with a reconciliation worker, and webhooks only once volume or latency makes polling genuinely expensive. Polling has no failure mode you have not already handled: your status endpoint is the source of truth, and a client that goes away leaves no dangling state.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Webhooks invert that trade. You gain delivery latency measured in milliseconds instead of your poll interval, and you pay with a public endpoint, HMAC verification, a replay window, deduplication, retry handling, and a reconciliation sweep anyway.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">They are worth it when the job is user-facing and long, when you deliver tens of thousands of completions per day, or when the consumer is a server you control. A gateway that normalises providers behind one endpoint lets you start with polling and add webhooks later without changing your client contract. <a href=\"https:\/\/qoraapi.com\/\">Qora API<\/a> is one option: one OpenAI-compatible key across GPT, Claude, Gemini and others, so your delivery logic survives a provider swap.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Frequently asked questions<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">How fast should a webhook endpoint respond?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Under five seconds, and ideally under 500 milliseconds. Verify the signature, insert the event id under a unique constraint, enqueue, return <code>200<\/code>. Anything else belongs in a worker. A handler that occasionally takes 8 seconds against a 10-second delivery timeout accumulates retries during spikes, precisely when you can least afford them.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Can I get exactly-once delivery?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">No, not end to end. At-least-once transport plus idempotent consumers is the achievable target, and it yields exactly-once effects as long as every side effect is keyed. Treat any claim of exactly-once delivery spanning a queue, a database and a third-party API as a description of deduplication.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">What if a provider never sends the webhook?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">You must detect it. Record the expected delivery deadline at submission and run a sweep for anything past it. If the provider reports the job finished, settle it locally and emit the event yourself. Without the sweep, a webhook lost during a deploy becomes a job stuck forever.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Should I use streaming or webhooks?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">They solve different problems. Streaming optimises time-to-first-token for a user watching output appear; webhooks optimise completion notification for a job nobody is watching. A long document analysis might stream partial sections and still fire a webhook when the structured result is ready.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">How do I choose a lease length?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Start from your p99 job duration and add margin, then add a heartbeat so the value need not cover worst-case runtimes. A 60-second lease with a 15-second heartbeat handles jobs of any length while keeping reclaim latency under a minute. Sizing the timeout to the worst case means a crashed worker blocks that job for its full duration.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Asynchronous delivery is not a feature you bolt onto an API; it is a different contract. Once submission and completion are separated you own a durable job row, a lease, a retry policy, an idempotency story and a reconciliation loop. In return, a 90-second generation stops being a failure and becomes ordinary.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Build the state machine first, get the lease and idempotency keys right, and start with polling. Add webhooks when you can point at a specific latency or cost problem that polling causes. Keep the reconciliation sweep running either way, because it turns an at-least-once world into a correct one. A unified gateway such as <a href=\"https:\/\/qoraapi.com\/\">Qora API<\/a> reduces that provider-specific surface to one endpoint and one billing model.<\/p>\n\n\n\n\n<h3 class=\"wp-block-heading\">Related reading<\/h3>\n\n\n<ul class=\"wp-block-list\"><li><a href=\"https:\/\/qoraapi.com\/blog\/batch-ai-api-processing\/\">Batch AI APIs: Processing Millions of Requests Affordably<\/a><\/li><li><a href=\"https:\/\/qoraapi.com\/blog\/ai-api-rate-limits-429-errors\/\">How to Handle AI API Rate Limits and 429 Errors<\/a><\/li><li><a href=\"https:\/\/qoraapi.com\/blog\/reliable-ai-agents\/\">Building Reliable AI Agents: Guardrails, Retries, and Human-in-the-Loop<\/a><\/li><li><a href=\"https:\/\/qoraapi.com\/blog\/idempotency-safe-retries-ai-api\/\">Idempotency and Safe Retries for AI APIs<\/a><\/li><\/ul>\n\n","protected":false},"excerpt":{"rendered":"<p>Synchronous request\/response breaks down for long generations. Compare streaming, polling and webhooks, then design the job lifecycle, queue, webhook consumer and idempotency correctly.<\/p>\n","protected":false},"author":1,"featured_media":293,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[5,6,9,7],"class_list":["post-294","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-api","tag-ai-api","tag-api-gateway","tag-developer-tools","tag-developers"],"_links":{"self":[{"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/posts\/294","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/comments?post=294"}],"version-history":[{"count":1,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/posts\/294\/revisions"}],"predecessor-version":[{"id":307,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/posts\/294\/revisions\/307"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/media\/293"}],"wp:attachment":[{"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/media?parent=294"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/categories?post=294"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/qoraapi.com\/blog\/wp-json\/wp\/v2\/tags?post=294"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}