Offline Mobile AI: Running On-Device Agents via Gemini Nano and ML Kit

When handling highly sensitive user operations—such as healthcare communications, financial computations, or enterprise field metrics—routing user data over an external network introduces security liabilities, compliance friction, and unpredictable data costs.

The solution is **On-Device Local Inference**. By using Google ADK backed by **Gemini Nano** and the **ML Kit Generative AI SDK**, developers can route processing tasks safely to native device silicon (CPU, GPU, and NPU). This guarantees absolute data privacy, completely bypasses network latencies, and allows features to function flawlessly **100% offline**.

Architectural Blueprint: Cloud vs. Edge Execution

Standard implementations lean heavily on cloud models (e.g., Gemini 2.5 Flash), which can introduce API costs and latency. On-device execution isolates the inference lifecycle inside the Android OS boundaries. Google AICore manages the underlying model provisioning, ensuring that updates happen seamlessly via Google Play Services without bloating your base APK size.

Step-by-Step Implementation: Configuring ML Kit for Local Execution

Let's configure a native Android engine to securely invoke Gemini Nano locally via ML Kit inside your clean architecture layer.

1. Configure Dependencies

Open your app-level build.gradle.kts and add the dedicated ML Kit Generative AI dependency wrapper alongside your Google ADK core components:

dependencies {
    // Core Google ADK Components
    implementation("com.google.adk:google-adk-kotlin-core-android:0.1.0")
    
    // Google ML Kit On-Device GenAI Core Client
    implementation("com.google.mlkit:generative-ai-android:16.0.0-alpha02")
}

2. Initializing Local Models via GenerativeModel

To run tasks offline, we leverage ML Kit's local inference entry point instead of a cloud model instance. Here is how to configure a secure, self-contained local agent repository helper module:

package com.example.adkdemoapp.security

import android.content.Context
import com.google.adk.kt.agents.Instruction
import com.google.adk.kt.agents.LlmAgent
import com.google.mlkit.vision.common.internal.MobileVisionBase
import com.google.mlkit.nl.genai.LocalGenerativeModel

object SecureLocalAgentProvider {

    private lateinit var localModelInstance: LocalGenerativeModel

    /**
     * Initializes the underlying Gemini Nano engine via AICore context constraints.
     */
    fun initializeLocalEngine(context: Context) {
        localModelInstance = LocalGenerativeModel.builder()
            .setModelName("gemini-nano") // Safely targeting local NPU/GPU boundaries
            .setContext(context.applicationContext)
            .build()
    }

    /**
     * Creates a highly private, decoupled on-device Agent
     */
    fun createPrivacyAgent(): LlmAgent {
        return LlmAgent(
            name = "local_privacy_agent",
            description = "This agent executes completely offline. No external data leaks are possible.",
            // Binding the local ML Kit hardware runner instance directly
            model = localModelInstance.toAdkModel(), 
            instruction = Instruction(
                """
                You are an offline, secure processing node.
                1. Process incoming information utilizing ONLY the local context provided.
                2. Never request information from an external resource.
                3. Mask all personally identifiable information (PII) automatically before formatting output data strings.
                """.trimIndent()
            )
        )
    }
}

3. Processing Tasks Locally via GenaiPrompt Loops

When the user triggers a sensitive action, we feed the contextual payload cleanly through the local agent's GenaiPrompt pipeline, ensuring zero remote transmission overhead:

package com.example.adkdemoapp.security

import androidx.lifecycle.ViewModel
import androidx.lifecycle.viewModelScope
import com.google.adk.kt.runners.AgentRunner
import kotlinx.coroutines.flow.MutableStateFlow
import kotlinx.coroutines.flow.StateFlow
import kotlinx.coroutines.launch

class SecureProcessingViewModel : ViewModel() {

    private val _processingState = MutableStateFlow<String>("Awaiting Input")
    val processingState: StateFlow<String> = _processingState

    fun analyzeSensitiveDataLocally(rawInputText: String) {
        viewModelScope.launch {
            _processingState.value = "Processing securely on-device..."
            
            try {
                // Initialize runner cleanly using the local agent node infrastructure
                val localRunner = AgentRunner.create(agent = SecureLocalAgentProvider.createPrivacyAgent())
                
                // Construct a contextual local prompt wrapper boundary
                val securePrompt = "Analyze and summarize this text locally, stripping out raw IDs: $rawInputText"
                
                var accumulatedResult = ""
                localRunner.executeStream(securePrompt).collect { chunk ->
                    accumulatedResult += chunk.textDelta
                }
                
                _processingState.value = accumulatedResult
            } catch (e: Exception) {
                _processingState.value = "Hardware processing error: ${e.localizedMessage}"
            }
        }
    }
}

Optimizing Latency & Memory Allocation

• Model Warm-Up: On-device NPUs can take a brief moment to spin up context constraints. Call `initializeLocalEngine(context)` early in your application lifecycle (such as inside a custom `Application` class) to completely eliminate first-run delay.
• Context Pruning: Gemini Nano is highly optimized for short, specialized tasks. Keep your local prompt injection boundaries lean and crisp to preserve system RAM footprints and minimize execution times.