Integrating AI APIs into Mobile Apps

Shipping AI in an iOS or Android app comes down to one architectural decision: the app never holds a provider API key. Put your own backend between the client and the model, stream over a resumable connection, degrade gracefully when the network drops, and meter every user server-side. This guide covers the patterns, the platform limits, and working code.

Three constraints shape everything below: the client is untrusted (it can be rooted, jailbroken or instrumented), the network is intermittent by default, and the OS suspends your process whenever it likes.

The cardinal rule: never ship provider API keys in the app

An APK or IPA is a zip archive. unzip -l, strings, jadx or Frida will surface a hardcoded key in seconds. Obfuscating it, splitting it across constants, computing it at runtime, or storing it in the Keychain/Keystore only raises the effort — it never makes the key unextractable, because the app has to read it to use it. Treat any secret the client can read as public.

The damage from a leaked key is not just someone else’s bill. Your quota is consumed, your rate limits are exhausted for real users, your organization is attached to abusive traffic, and remediation means rotating a credential that is compiled into an app version already installed on thousands of devices. You cannot rotate faster than your users update. A backend proxy lets you rotate in seconds.

The same applies to a gateway key: a relay credential is a provider credential with a different label, and putting it in the binary just relocates the problem. The one legitimate exception is deliberate BYOK. Store a user-supplied key in the platform secure store (iOS Keychain with kSecAttrAccessibleWhenUnlockedThisDeviceOnly, Android EncryptedSharedPreferences backed by the Keystore), never log it, and accept that on a compromised device the blast radius is the user’s own account — which is the point.

Asset	Where it lives	Rotation
Provider / relay API key	Server secret manager, injected as env var	Centrally, no app release
User session token	iOS Keychain / Android EncryptedSharedPreferences	Short TTL plus refresh token
Per-user quota counters	Server (Redis or database), keyed by user id	n/a
Cached AI results	Encrypted app storage, scoped per user, TTL	Cleared on logout
BYOK user key (optional)	Platform secure store, device-only accessibility	User-initiated

Architecture: app to your backend to the AI relay

The shape is a thin proxy with four responsibilities:

• Authenticate the caller. The device presents a short-lived access token (OAuth 2.0 / OIDC, refreshable) and your backend resolves it to a user id. Never trust a user id sent in the request body.
• Hold the credentials. Provider or relay keys live in a secret manager and are read at boot. The client never sees them.
• Enforce policy before forwarding. Per-user rate limits, token budgets, model allow-lists, output caps and content rules. Rejecting a request is far cheaper than generating a response and then rejecting it.
• Normalize and log. One OpenAI-compatible request shape upstream, one usage record per call downstream (user, model, input tokens, output tokens, latency).

Point the upstream hop at a single OpenAI-compatible endpoint rather than five provider SDKs. One base URL and one credential means your client contract never changes when you swap models, and there is exactly one secret to rotate — which matters when the rotation is triggered by an incident rather than a roadmap. That is the role of an AI API relay: qoraapi.com exposes many models behind one OpenAI-compatible endpoint. If the proxy layer is new to you, start with our guide on how to integrate an AI API.

// Server-side proxy: holds the key, attaches identity, enforces quota, streams back.
import express from "express";
const app = express();
app.use(express.json({ limit: "1mb" }));

const RELAY = process.env.AI_RELAY_BASE_URL;   // never shipped to the client
const KEY   = process.env.AI_RELAY_API_KEY;

app.post("/v1/generate", requireSession, async (req, res) => {
  const { userId } = req.session;              // from the verified token, not the body
  const { model, messages, max_tokens = 512 } = req.body;

  if (!(await takeTokens(userId, max_tokens))) // atomic Redis counter, pre-debit
    return res.status(429).set("Retry-After", "30").json({ error: "quota_exceeded" });

  const upstream = await fetch(`${RELAY}/v1/chat/completions`, {
    method: "POST",
    headers: { "Authorization": `Bearer ${KEY}`, "Content-Type": "application/json" },
    body: JSON.stringify({ model, messages, max_tokens, stream: true,
                           user: userId })     // stable id for provider-side abuse signals
  });

  res.set({ "Content-Type": "text/event-stream", "Cache-Control": "no-cache",
            "X-Accel-Buffering": "no" });      // stop nginx from buffering the stream
  await pipeWithUsageAccounting(upstream.body, res, { userId, model });
});

Two details there are load-bearing. Pre-debiting the quota before the upstream call prevents a race where parallel requests each pass the check; and X-Accel-Buffering: no is the difference between token-by-token output and one blob delivered at the end when you sit behind nginx. The wire format itself is covered in our AI API streaming guide.

Streaming to mobile over flaky networks

Token streaming is what makes an AI app feel fast, and it is the feature most likely to break on cellular. Three mobile-specific failure modes dominate:

• Idle NAT timeouts. Carrier and home NAT tables drop connections that go quiet for roughly 30–60 seconds, so a long pause before the first token kills the socket.
• Network handoff. A Wi-Fi to LTE transition replaces the local IP. The socket is dead but the app is not told; TCP hangs until a write fails.
• Process suspension. Backgrounding the app suspends the process and the socket dies with it, usually with no error callback.

The fixes are concrete. Have the backend emit an SSE comment heartbeat (: ping) every 15 seconds so the connection never goes idle, and treat two consecutive missing heartbeats as a dead connection instead of waiting for a timeout. Never apply a normal read timeout to a streaming request — set it to zero on the client and rely on heartbeats plus a server-side overall deadline for liveness. Reconnect with exponential backoff plus jitter, capped at a handful of attempts so an offline device does not spin the radio. And make the stream resumable.

Resume works like this: the server tags every event with a monotonically increasing id, the client remembers the last id it applied, and on reconnect it sends Last-Event-ID. Your backend keeps a short-lived buffer of emitted events per generation id — a few minutes is plenty — and replays from the requested offset. Because a replay can overlap what the client already rendered, dedupe on event id before appending, or a reconnect duplicates text mid-sentence.

Transport	Best for	Mobile failure mode	Verdict
SSE over HTTP	One-way token streaming	Idle NAT kills; buffering proxies	Default for chat and completions
WebSocket	Bidirectional mid-stream control (interrupt, tool approval, voice)	Reconnect and state are entirely yours; worse under backgrounding	Only when you need true duplex
Long-poll	Legacy clients, proxies that break SSE	Battery and data cost per poll	Fallback only
Async job + push	Anything over ~20–30s	None — no socket is held	Required for image, video and agent runs

// Android (OkHttp): resumable SSE with heartbeats, backoff and jitter.
private val client = OkHttpClient.Builder()
    .connectTimeout(10, TimeUnit.SECONDS)
    .readTimeout(0, TimeUnit.MILLISECONDS)   // streams are long-lived: no read timeout
    .retryOnConnectionFailure(false)         // we do our own resume
    .build()

suspend fun stream(generationId: String, body: RequestBody, token: String,
                   onToken: (String) -> Unit) = withContext(Dispatchers.IO) {
    var lastEventId: String? = null
    val seen = mutableSetOf<String>()        // dedupe replayed events
    for (attempt in 0 until 6) {
        val req = Request.Builder()
            .url("$BASE/v1/generate/$generationId")
            .header("Authorization", "Bearer $token")
            .header("Accept", "text/event-stream")
            .apply { lastEventId?.let { header("Last-Event-ID", it) } }
            .post(body)
            .build()
        try {
            client.newCall(req).execute().use { res ->
                if (res.code == 429 || res.code >= 500) throw IOException("retry ${res.code}")
                res.body!!.source().use { src ->
                    while (!src.exhausted()) {
                        val line = src.readUtf8Line() ?: break
                        when {
                            line.startsWith("id: ")   -> lastEventId = line.removePrefix("id: ")
                            line.startsWith("data: ") -> {
                                val data = line.removePrefix("data: ")
                                if (data == "[DONE]") return@withContext
                                if (seen.add(lastEventId ?: data)) onToken(parseDelta(data))
                            }
                            line.startsWith(": ping") -> Unit   // heartbeat, connection alive
                        }
                    }
                }
                return@withContext                    // clean EOF
            }
        } catch (e: IOException) {
            if (attempt == 5) throw e
            delay((500L shl attempt) + Random.nextLong(0, 400))  // backoff + jitter
        }
    }
}

iOS is the same protocol with different primitives: URLSession.shared.bytes(for:) gives you an AsyncSequence of lines, and you re-issue the request with the Last-Event-ID header inside an attempt loop. Do not use a background URLSession configuration for SSE — it is built for file transfers and will deliver a response to a process that no longer exists.

The moment your app backgrounds, stop pretending the stream will survive. Persist the generation id, close the socket cleanly, and either resume on foreground with Last-Event-ID or hand the work to the async job pattern below. Keeping partial text matters: if the stream died at 60%, show what you have, mark it incomplete, and offer “continue”. Users tolerate latency far better than they tolerate losing text.

Offline and degraded modes

A mobile AI feature should assume it will sometimes run with no usable network. Design three states explicitly instead of letting the UI collapse into an error toast.

• Queue, don’t drop. Persist outbound requests in a durable local queue (Room/SQLite on Android, Core Data or SQLite on iOS) with a client-generated idempotency key. On reconnect, drain the queue; the server deduplicates on that key, so a retry after a timeout never double-charges quota or creates two generations.
• Optimistic UI with an explicit state machine. Render the user’s message immediately as queued and move it through sending → streaming → complete | failed. The user gets instant feedback and you get one place to reason about retries.
• Cache the last result. Key the cache on a hash of the normalized prompt plus model and parameters, store it encrypted with a TTL, and serve it offline with a visible “cached” label. Never present stale output as fresh, and clear per-user caches on logout.

Make the degraded decision measurable rather than a guess: log whether each request streamed or not alongside the network type and battery band, and after a few weeks you will know exactly which conditions justify turning streaming off. The rule of thumb that survives contact with data is simple — when bytes and radio time matter more than time-to-first-token, drop streaming.

Platform specifics: iOS and Android are not the same problem

The OS imposes hard limits you cannot engineer around, and they differ enough that “mobile” is not one target.

iOS

• Background execution is measured in seconds, not minutes. A beginBackgroundTask window is short and finite; BGProcessingTask and BGAppRefreshTask are scheduled opportunistically by the system and are useless for a response the user is waiting on.
• When the app is suspended its sockets are closed. No configuration keeps an SSE connection alive in a suspended app.
• Silent (content-available) pushes are throttled and best-effort. For a finished long-running job, send a visible notification with a deep link into the result rather than relying on a silent push to wake the app and fetch.

Android

• A foreground service with a persistent notification is the sanctioned way to keep work alive. Since Android 14 you must declare a foregroundServiceType (for example dataSync or shortService) and hold the matching permission. Misusing it is a policy problem, not just a technical one.
• WorkManager with expedited work is the right tool for “finish this later and notify me” — it survives process death and respects Doze.
• In Doze and App Standby, network access is deferred. A high-priority FCM data message buys a brief wake-up window; use it to schedule work, not to run a long generation inline.

Concern	iOS	Android	Recommended pattern
Background socket survival	Suspended in seconds; sockets closed	Killed under Doze / App Standby	Never hold a socket in the background
Sanctioned long work	BGProcessingTask; background URLSession (files only)	Typed foreground service; expedited WorkManager	Async job plus notification
Waking the app	APNs (silent push unreliable)	High-priority FCM	Wake to schedule, then fetch
Battery and data lever	Coalesce requests; avoid polling	Same, plus metered-network awareness	One batched call beats five small ones

Cost and abuse control on mobile

Every control implemented in the client is a UX affordance, not a security control. An attacker with a proxy and a patched build bypasses all of it. The server is the only place enforcement counts.

• Device attestation. Apple’s App Attest and Google’s Play Integrity produce a signed assertion that your server verifies, binding a device key to your app’s identity. Gate the most expensive endpoints on it rather than the whole app: rooted devices, emulators, custom ROMs and sideloaded builds legitimately fail, and locking them all out costs you real users. Verify the assertion server-side — a client that merely reports “I am attested” has told you nothing.
• Per-user quotas in tokens, not requests. One long prompt can cost orders of magnitude more than a short one, so a request-count limit is trivially evaded. Track input and output tokens separately, reset on a rolling window, and return 429 with a Retry-After the UI can surface as a real message.
• Anomaly signals. Watch requests per device per minute, distinct devices per account, near-identical prompt templates across accounts, and sudden shifts in the app-version or platform mix. One account issuing thousands of requests from a single emulator fingerprint is a cheap signal to catch.
• Usage records you can bill from. Log user id, model, input tokens, output tokens, latency and a request id per call. Keep the price table in configuration, not in code, so a price change is a config edit — the same discipline we describe in our guide to metering AI usage.
• BYOK for power users. Letting users bring their own provider key moves cost and rate limits off your books, and it is the honest option for genuinely high-volume users. The tradeoff is that support now includes their key problems.

Long-running jobs: async plus push, not a held connection

Anything that routinely exceeds 20–30 seconds — image or video generation, agent runs, batch summarization, long document analysis — should not be a held stream. A held connection burns battery, dies on every handoff, and occupies a concurrency slot on your backend for the whole duration, which is precisely the resource that limits how many users you can serve.

The pattern: the client POSTs to /jobs, the backend returns 202 Accepted with a job id, and the result arrives by push (FCM/APNs) with a deep link. The client also polls with exponential backoff while it is in the foreground, because push delivery is best-effort and polling is what makes the feature work on a bad network. Two non-obvious requirements: send an idempotency key on job creation so a retry after a client timeout does not create a second job, and persist the job id locally so a reinstall does not orphan work the user already paid for.

If the workload is genuinely bulk — hundreds or thousands of items — do not loop over the interactive endpoint at all. Batch endpoints accept many items in one submission, run them asynchronously at a lower unit cost, and are built for exactly this shape; our batch AI API processing guide covers submission, polling and result retrieval.

Frequently asked questions

Can I obfuscate the API key instead of building a backend?

No. Obfuscation raises the cost of extraction; it does not prevent it, because the app must read the key at runtime. You also lose the ability to rotate the credential without shipping a release and waiting for users to update. The backend proxy is the only design that lets you revoke a leaked key in seconds.

Should I use SSE or WebSockets for mobile streaming?

SSE for one-way token streaming, which is almost every AI feature. It rides normal HTTP, passes through most proxies and CDNs, and reconnects with a well-defined resume mechanism (Last-Event-ID). Choose WebSockets only when you genuinely need bidirectional mid-stream control such as user interruption or tool approval, and be prepared to manage reconnect state yourself.

How do I keep a generation running when the user leaves the app?

You move it off the connection. Give the generation a server-side job id, let the backend continue (or submit it to a batch endpoint), and notify the device with a push notification when it finishes. Holding a socket in a backgrounded iOS app is impossible, and on Android it requires a foreground service with a persistent notification — a heavy price for a chat reply.

Does device attestation actually stop abuse?

It raises the bar significantly but is not a wall. It blocks casual scripted abuse and emulator farms; it does not stop a determined attacker with a rooted device or a patched client. Use it as one layer alongside server-side token quotas, per-account anomaly detection and rate limits — never as the only control.

Conclusion

The mobile AI architecture that survives production is small and boring on the client: no keys, a resumable stream, a durable queue, and explicit degraded states. The complexity lives on your backend, where you control rotation, quotas and metering. Build the proxy first — a day of work that saves you an incident — then add streaming with resume, then move every long job to async plus push.

To go deeper, read the guide on how to integrate an AI API for the request and response fundamentals, the AI API streaming guide for the wire format, and metering AI usage for the billing side. If you want one endpoint and one credential in front of every model your app calls, the proxy layer in this article is the only integration you need to write.

Related reading

• How to Integrate an AI API into Your Application
• AI API Streaming Explained: How SSE Works and How to Consume It
• Metering and Billing AI Usage Per User: A Practical SaaS Guide
• How to Build an AI Chatbot with the API