On-Device AI in Mobile Apps: What You Can Build Without Cloud AI (and When You Still Need It)

On-device AI runs models on the phone, which means features work offline, respond instantly, keep data on the device and cost nothing per request. In 2026 both platforms provide built-in language models: Apple's Foundation Models framework (on Apple Intelligence devices) and Gemini Nano through ML Kit's GenAI APIs and Prompt API (on supported Android devices), plus Core ML and LiteRT for custom models. They are excellent for focused tasks (summarizing, extracting fields, classifying, rewriting, tagging images) and weak at broad knowledge, long context and complex reasoning. Most apps should use a hybrid design: on-device first where it is good enough and the device supports it, with a cloud model or non-AI path as fallback.

Cloud AI is powerful but has three costs that matter in mobile products: every request costs money, every request needs a network connection, and every request sends user data to a server. On-device models remove all three for the tasks they can handle.

Apple. The Foundation Models framework is a Swift API that gives apps direct access to the on-device model that powers Apple Intelligence. Apple's iOS 27 guidance adds multimodal prompts (images with text), on-device Vision tools such as OCR and barcode reading that the model can call, Dynamic Profiles to switch models, tools and instructions within a session, and an Evaluations framework for testing AI features. The framework now also accepts other models through a common protocol, including cloud models, so one API can route between on-device and server models. Apple also offers its larger models on Private Cloud Compute, at no cloud API cost for developers in the App Store Small Business Program with fewer than two million first-time downloads.

Android. ML Kit's GenAI APIs provide task-specific features (such as summarization, proofreading, rewriting and image description) and a Prompt API for custom prompts, running Gemini Nano through AICore on supported devices. Google announced at I/O 2026 a Structured Output API for the Prompt API, prefix caching, and Gemini Nano 4 in developer preview ahead of flagship devices later in 2026. Firebase AI Logic supports hybrid inference with explicit modes to prefer or require on-device or cloud models.

Custom models. For specialist tasks (defect detection, document classification, wake words), Core ML on iOS and LiteRT on Android run models you train or adapt yourself.

Platform on-device language models are small compared with frontier cloud models. That shapes what to use them for.

Few apps should be all on-device or all cloud. Common patterns:

Platform language models only run on recent devices with AI features enabled, in supported languages and regions, and models may still be downloading after a user enables them. Your app must check availability at runtime and handle every outcome: available, not supported, not enabled, not ready yet. Look at your own analytics to see what share of active users have capable devices; for many consumer apps in price-sensitive markets it will be a minority for some time.

That makes the fallback design a product decision, not an afterthought. Decide whether users without on-device support get a cloud version (with its cost and privacy implications), a simpler non-AI version, or no feature.

On-device inference has no API bill, but it uses memory, processor time and battery. Keep prompts and outputs short, avoid running models in tight loops, batch background work, and measure on real low-end supported devices rather than the newest phone on the team. For custom models, size and quantization matter for download size and memory; our AI edge deployment guide covers model compression.

On the development side, budget for testing across devices and OS versions, evaluating output quality with realistic data, and building and maintaining the fallback path.

On-device processing is a genuine privacy advantage and worth stating clearly in your product and privacy policy. It is not a complete answer. Results can still be stored, synced, logged or sent to analytics, and hybrid features may send some data to the cloud. Be precise in what you tell users about which features run where. See mobile app data privacy.

React Native and Flutter apps can use platform models through native modules or plugins, and custom models through cross-platform runtimes. Because Apple's and Google's models differ in capability, prompt behaviour and availability, build a thin abstraction (a `summarize` or `extractFields` function) with platform-specific implementations, and evaluate output quality on each platform separately. See React Native app development.

On-device AI is now a practical choice for focused features that need to be fast, private, offline-capable or affordable at volume. It is not a replacement for cloud models, and device coverage limits who gets it. Design narrow tasks with structured outputs, route intelligently between device and cloud, and treat the fallback as part of the product. For the wider picture of AI in mobile apps, see AI-powered mobile app development, and to let assistants use your app's features, see App Intents and AppFunctions.