Design an analytics SDK (Android and iOS): queue, batching, retry and consent

How to read this. A library question with a data pipeline inside it. Sections 1 to 9 use Kotlin; section 10 maps to iOS.

TL;DR. The SDK is a durable, bounded outbox behind a thin public API. track validates, stamps an event ID and client time, and hands the event to one background worker that writes it to disk. A flush loop reads a stable batch, sends it with a batch ID, and deletes it only after the server accepts it. Retries are safe because event IDs and batch IDs let the server dedupe. Never block the host app, never crash it, never collect before consent, and cap the queue with a documented drop policy.

1. Clarify the requirements (about 5 min)

Ask the app developers:

• Do hosts embed other SDKs, and is there a startup budget? (Assume yes to both: init does no I/O.)
• Which consent regimes apply? (Assume EU and US states, so collection must be gateable and erasable.)

Ask the data team:

• Is a duplicate worse than a lost event? (Assume at-least-once delivery with server dedupe on event ID.)
• Do they need ordering, and can they tolerate wrong clocks? (Per-device order by sequence number; no, so we send two timestamps.)

Non-functional:

Out of scope: the backend pipeline, attribution, crash reporting.

2. High-level architecture (about 10 min)

Two decisions carry the design. One writer: all store access goes through a single worker (a single-threaded dispatcher on Android, an actor on iOS), not scattered locks. Disk is the source of truth: events leave the store only after the server confirms the batch, and flush policy is a small object driven by remote config.

Official architecture mapping: The SDK sits beside the host's UI, optional domain and data layers; host repositories or ViewModels call it, never composables, and it needs only the Application context. Inside, the public API is the facade, the SQLite queue is the single source of truth, and the uploader drains it as queued writes.

3. Public API and data model

Keep the surface small. Every public method is a contract for years.

// Sketch: public surface (Kotlin)
class Analytics private constructor(/* internal wiring */) {
    companion object {
        fun init(context: Context, config: AnalyticsConfig): Analytics   // cheap; no I/O; keeps only applicationContext
    }
    fun track(name: String, properties: Map<String, Any?> = emptyMap())   // main-safe, non-suspend
    fun screen(name: String)
    fun identify(userId: String, traits: Map<String, Any?> = emptyMap())
    fun flush()                       // a request, returns immediately
    fun reset()                       // new anonymous ID, clear user ID
    fun setConsent(consent: Consent)  // GRANTED, DENIED, UNKNOWN (default)
    fun optOut()                      // stop collecting, drop the queue
}

Choices to defend:

• No return value that invites waiting. track returns Unit; an awaitable flush lives in the test module.
• Properties are a restricted map of strings, numbers, booleans, null and bounded nested maps and lists. Reject the rest at the boundary.

The stored event:

Validation. Check name format, property count, value types and encoded size. In release, drop the event and count it by reason; in debug, log loudly. Never throw to the caller.

4. Threading and startup

The host's main thread is not ours.

• track allocates and hands off without blocking: a bounded Channel with trySend on Android, AsyncStream.Continuation.yield on iOS.
• Wrap every entry point in a catch-all. A bug in our code becomes a dropped event and a counter, not a host crash.

Auto-init. Many SDKs initialize through a ContentProvider so hosts write no code. The App Startup docs say content providers are expensive to instantiate, can slow startup, and initialize in an undetermined order. App Startup shares one InitializationProvider and supports lazy initialization with AppInitializer.initializeComponent after disabling automatic init with tools:node="remove". For an SDK the better default is explicit: the host calls Analytics.init and the SDK ships no provider. An SDK that collects before consent is a privacy bug, not a convenience.

5. The hard part: queue, batching, retry and clocks

Durable queue

Use SQLite directly or Room as an internal detail. One table for events, one for batches.

// Sketch: write path and flush (Kotlin, simplified, error handling trimmed)
internal class Engine(
    scope: CoroutineScope,                 // SDK-owned external scope; injected single-threaded dispatcher + SupervisorJob
    private val store: EventStore,
    private val uploader: Uploader,
    private val policy: FlushPolicy,
) {
    private val inbox = Channel<Event>(capacity = 1_000, onBufferOverflow = BufferOverflow.DROP_OLDEST)

    init {
        scope.launch {
            for (event in inbox) {
                runCatching { store.append(event) }
                if (store.pendingCount() >= policy.flushAt) flushOnce()
            }
        }
    }

    fun enqueue(event: Event) { inbox.trySend(event) }   // never suspends, never throws

    suspend fun flushOnce() {
        val batch = store.leaseBatch(policy.maxBatchSize) ?: return   // reuses an open lease first
        when (uploader.send(batch)) {                                 // enum SendResult
            SendResult.OK, SendResult.REJECTED -> store.delete(batch.id)
            SendResult.RETRY -> store.release(batch.id)               // same batch next time
        }
    }
}

The detail that matters is leaseBatch. The first time events are selected, the store assigns a batchId and records which event IDs belong to it. A retry resends the same batch with the same ID and events, even if newer events arrived. The server can then ignore any batch ID it has seen.

Crash safety. SQLite commits are atomic, so a half-written row never ships. Committing every event is safest and slowest; grouping appends into one transaction is the usual compromise, and a kill before the commit loses that group. State the trade-off.

Flush policy

N, T and the batch cap come from remote config with bundled defaults; any number you quote is illustrative.

Do not rely on the process staying alive. This is a queued-write design: the official offline-first term for draining a queue with exponential backoff via WorkManager, which the docs list analytics events as a fit for. Work lifetimes map as follows: track is app-oriented (the engine's external scope, not the caller's); so are the background flush and in-process retry; the periodic flush and retry after process death are business-oriented, so WorkManager owns them (a unique CoroutineWorker with a network Constraints.Builder, enqueueUniquePeriodicWork for the periodic one, and Result.retry() for failures). The worker only calls flushOnce, since the queue is the source of truth. Its docs say Result.retry() reschedules by the backoff policy (default exponential, 30 second delay, 10 second minimum), and run time depends on constraints and system optimizations, so promise no delivery time.

Backoff and retry

• Retry network errors, timeouts, 429 and 5xx; honor Retry-After.
• Do not retry a 400. Delete the batch and count it; a poison batch must not block the queue.
• Use exponential backoff with jitter and a cap, one upload in flight, and a persisted failure count.

Idempotency. The server dedupes on event ID within a retention window and can short-circuit on batch ID. Exactly-once is not on offer; at-least-once plus dedupe is.

Clock skew

Device clocks are wrong. Send clientTime on each event and sentAt on the batch, both from the device clock; the server records receivedAt. Then receivedAt - (sentAt - clientTime) estimates the true event time, assuming a constant offset and small latency. A monotonic delta per event keeps in-process order if the user changes the clock.

Backpressure and drop policy

• Inbox: bounded, drop oldest.
• Disk: cap by count and bytes; when full, drop the oldest.
• A per-name rate limit and size cap stop a runaway loop in app code.
• Expose counters: drops by reason, queue depth, oldest event age.

Dropping oldest favors fresh data; dropping newest keeps a coherent prefix. Document the choice.

6. Network, sessions and sampling

Network. Send one gzip-compressed body per batch, with shared context (app, OS and SDK version) once. Keep call count low, since radio wakeups cost more than bytes. Treat the write key as public.

Sessions. A new session starts at first launch or when the app returns to the foreground after a configurable idle gap, stamped at track time because the SDK sees lifecycle events.

Identity. The anonymous ID is a random UUID, not derived from hardware. identify labels later events only; the server stitches identities.

Sampling. Decide once per install or session from a hash of the anonymous ID against a remote-config rate, and stamp the rate on events so analysts can reweight. Per-event sampling breaks funnels.

7. Privacy and consent

Make the safe behavior the default.

• No collection before consent. Consent is UNKNOWN, GRANTED or DENIED; UNKNOWN acts like DENIED. Persist it, and never write pre-consent events to disk.
• Opt-out must act: stop collecting, delete the queue, and offer a deletion hook for erasure requests.
• Data minimization: no advertising ID by default, no fingerprinting, no automatic text capture. Scrub known PII keys and accept a host redaction callback.
• Advertising ID on Android. The docs say users can reset it and opt out, and apps should not cache it. For apps targeting Android 13 (API 33) or higher, the Play services ID needs the AD_ID permission or the ID is replaced with zeros, and an SDK that declares the permission has it merged into the host manifest. An SDK that does not need the ad ID should not declare it.
• ATT on iOS. Apple says an app must use App Tracking Transparency if it collects data about people and shares it with other companies to track them across apps and websites. The host owns the prompt, not the SDK.
• Disclosure. Google Play says developers must reflect data collected or shared through third-party SDKs in the Data safety form, so ship a field-by-field description.

GDPR and CCPA compliance is the host's responsibility; the SDK provides the controls.

8. Versioning, remote config and multiple SDKs

Versioning. Semantic versions cover the public API and the wire format. Put schemaVersion on events and sdkVersion on batches; the server must accept old versions while old apps are in the field. Behavior is API: changing a default flush interval or drop policy changes someone's data volumes.

Remote config. A small document fetched at launch carries flush limits, sampling rate and a kill switch. Cache the last good copy, bundle defaults, and limit it to documented keys.

Multiple SDKs. Chain the uncaught-exception handler, namespace your database and work names, and avoid heavy transitive dependencies.

9. Testing and debuggability

Use constructor injection for the clock, dispatcher, network and store (no Hilt forced on the host; if the host uses @HiltWorker, give it a WorkerFactory), and ship FakeEventStore and FakeUploader in a test module, since official guidance prefers fakes to mocks. Test a kill between write and send, duplicate delivery and a clock jump. For debugging, offer a debug flag, a queue dump and a counters snapshot. Android's logging guidance says to minimize sensitive output and strip verbose logs from release builds, so keep logging off by default and redact values.

10. How to build an analytics SDK on iOS (Swift)

The architecture carries over. The differences are lifecycle, background time and the privacy manifest.

// Sketch: iOS facade (Swift 5.9+, iOS 17+ for AsyncStream.makeStream)
public enum PropertyValue: Sendable { case string(String), number(Double), bool(Bool), null }

struct Event: Sendable {
    let id: String, name: String, clientTime: Date
    let properties: [String: PropertyValue]
}

public final class Analytics: Sendable {
    private let inbox: AsyncStream<Event>.Continuation

    public init(config: AnalyticsConfig) {
        let (stream, continuation) = AsyncStream.makeStream(
            of: Event.self, bufferingPolicy: .bufferingNewest(1_000))
        self.inbox = continuation
        let engine = Engine(config: config)
        Task.detached(priority: .utility) { await engine.run(stream) }
    }

    public func track(_ name: String, properties: [String: PropertyValue] = [:]) {
        inbox.yield(Event(id: UUID().uuidString, name: name,
                           clientTime: Date(), properties: properties))   // sync, ordered
    }
}

actor Engine {
    init(config: AnalyticsConfig) { /* store opened lazily */ }
    func run(_ events: AsyncStream<Event>) async {
        for await event in events { /* append to SQLite, maybe flush */ }
    }
}

What is genuinely different:

• Privacy manifests are part of the SDK. Apple says a third-party SDK can ship PrivacyInfo.xcprivacy recording the data types it collects and the required reasons APIs it uses. It should include one if it uses a required reasons API, collects data about people using the host app, enables the app to collect such data, or contacts tracking domains. An SDK cannot rely on the manifests of the app or other SDKs.
• Tracking domains can fail. If NSPrivacyTracking is true, the manifest lists NSPrivacyTrackingDomains, and Apple says requests to them fail without tracking permission. Treat that as a normal network failure.
• Background time is finite. A beginBackgroundTask task must be ended before time expires or the system kills the app, and the expiration handler runs on the main thread, so keep it tiny. For work that must outlive the process, use BackgroundTasks.

11. Failure modes and edge cases

12. What changes by level

• Mid-level: a facade, a SQLite queue, batching by count and time, a background thread, simple retry, opt-out.
• Senior: batch leases with event and batch IDs, jittered backoff, a counted drop policy, clock-skew timestamps, lazy init, consent gating, never-crash wrappers.
• Staff: remote config with a kill switch, sampling, privacy manifest and disclosure for both stores, schema governance, SDK health telemetry, wire-format migration, build versus buy.

13. Follow-up questions to expect

1. How do you avoid duplicates? (Not at the client: at-least-once plus server dedupe.)
2. What if the clock is a year off? (Two timestamps and a monotonic delta; the server corrects.)
3. What if the user declines consent? (Nothing stored; earlier data is deleted on opt-out.)
4. How do you debug a data gap? (Drop counters by reason, queue age, last accepted batch ID, delivery rate per SDK version.)

Further reading (official docs)

• App Startup (Android): the shared InitializationProvider, lazy initialization, disabling auto-init
• WorkManager getting started (Android): Result.retry(), timing caveats
• Define work requests (Android): constraints, backoff policy and delays
• Guide to app architecture: recommendations (Android): DI, fakes over mocks, Context
• Data layer (Android): work lifetimes, external scope, single source of truth
• Offline-first (Android): queued writes, exponential backoff
• Advertising ID (Android): reset, opt-out, caching
• Behavior changes for Android 13: AD_ID
• Safeguard logs and sensitive data (Android)
• Provide information for Google Play's Data safety section: third-party SDK disclosure
• App Tracking Transparency (Apple): when the framework applies
• Privacy manifest files (Apple): SDK requirements, top-level keys, tracking domains
• Describing use of required reason API (Apple): per-SDK declarations
• beginBackgroundTask(withName:expirationHandler:) (Apple): finite time, expiration handler
• waitsForConnectivity (Apple): behavior when offline

Keep reading

• Design a push notification SDK
  Token lifecycle, delivery, deep links.
• Design a chat app
  Outbox, idempotent sends, WebSocket and push sync.
• Design a news feed
  Pagination, optimistic writes, cache invalidation.
• Guide: Staff-level mobile system design: what the round tests
  How staff answers differ from senior ones, what interviewers can observe, and a four-week practice plan.
• Guide: Offline-first mobile system design: one model, six case studies
  The official offline-first model applied to chat, feed, upload, booking, analytics and push.
• Guide: Native vs cross-platform: write the decision record
  A staff-level ADR for a framework bet: criteria, the assumption that flips it, migration, parity tests, orphaned libraries, rollback, and a free template.