Design a video streaming app (Android and iOS): adaptive streaming, preload and offline

TL;DR. Stream adaptive formats (HLS or DASH) and let the player pick the bitrate; your job is to constrain it, not replace it. Own one player outside the UI lifecycle so rotation, picture-in-picture and background audio do not restart playback. Spend bandwidth deliberately: preload only a few seconds of the next video, cap the disk cache, and drop quality on cellular or data saver. Measure time to first frame and rebuffer ratio, because those two numbers tell you whether any of this works.

1. Clarify the requirements (about 5 min)

Questions worth asking:

• Short feed videos, long-form, or both? (Assume both: an autoplaying feed and a watch screen.)
• Live or on demand only? (Assume on demand; live is a follow-up.)
• Offline downloads? (Yes, with DRM-protected content.)
• Resume across devices? (Yes.)
• Background audio and picture-in-picture? (Yes for long-form.)

Non-functional (the mobile part):

Out of scope: recommendations, upload and transcoding, ads. Say it out loud.

2. High-level architecture (about 10 min)

Two rules drive the design. The player is not owned by a screen: a player tied to an Activity restarts the video on rotation or backgrounding. Metadata and playback are separate paths: the feed API returns small JSON; the heavy bytes come from the CDN through the player.

Official architecture mapping: Compose screens plus ViewModels form the UI layer and VideoRepository with Room and REST data sources forms the data layer, with ViewModels never touching data sources. The architecture guide does not mention the Player; Media3 puts the Player and MediaSession in a MediaSessionService, and the UI only holds a MediaController. See architecture recommendations and background playback.

The watch ViewModel exposes one immutable uiState StateFlow (WhileSubscribed(5_000)), collected with collectAsStateWithLifecycle. It holds no Context, so the UI builds the MediaController.

3. Data model and API

Local schema (Room or SwiftData sketch)

@Entity(tableName = "videos")
data class VideoEntity(
    @PrimaryKey val id: String,
    val title: String,
    val thumbnailUrl: String,
    val durationMs: Long,
    val manifestUrl: String?,          // fetched lazily
)

@Entity(tableName = "playback_progress")
data class ProgressEntity(
    @PrimaryKey val videoId: String,
    val positionMs: Long,
    val updatedAt: Long,               // server-assigned when synced
    val dirty: Boolean,                // changed locally, not yet uploaded
)

Downloads are tracked separately (state, bytes, licence expiry). The repository maps entities to immutable models, with Flow for observed data.

REST

GET /v1/feed?cursor=<opaque>&limit=10&fields=id,title,channel.name,thumb,durationMs
GET /v1/videos/{id}/playback   → { manifestUrl, drm: {type, licenseUrl}?, resumeMs }
PUT /v1/videos/{id}/progress   → { positionMs, deviceId, clientTime }

Three details that matter:

• Field selection on the feed. A feed row needs a thumbnail, title, channel and duration. It does not need descriptions, full channel objects, caption lists or manifest URLs. Let the client name the fields (fields=, or a GraphQL selection) so each client asks only for what it renders, and a list of ten items stays small.
• Manifest on demand. Fetch /playback when a video is about to play or be preloaded, not for every row. Signed URLs expire, so fetching late avoids stale links.
• Progress as a PUT of an absolute position. It is idempotent, so retries are safe.

4. Adaptive streaming and the player (the core of the problem)

HLS and DASH

Both split a video into short segments at several bitrates and describe them in a manifest. HLS uses an .m3u8 playlist; DASH uses an .mpd. Apple documents HLS as supporting multiple alternate streams at different bit rates and switching in response to network bandwidth changes. Media3 plays both: MediaItem.fromUri(uri) is enough, and for a multivariant HLS playlist or a DASH manifest the player adapts between variants based on bandwidth and device capabilities. On iOS you play HLS, so a cross-platform catalogue usually ships HLS and adds DASH only if it needs to.

Startup time versus quality. A low first bitrate gives a fast but blurry start; a high one risks a stall. Do not write your own selection algorithm. Tune the inputs: a small lowest rung, short segments, and quality caps by network type (section 6).

One player, owned outside the UI

On Android, put the ExoPlayer and a MediaSession inside a MediaSessionService. Media3 documents this as the way to keep playing when the app is not in the foreground; the service needs the mediaPlayback foreground service type, and the UI controls the player through a MediaController.

// Sketch: player owned by a service, backed by a size-capped cache
class PlaybackService : MediaSessionService() {
    private var session: MediaSession? = null

    override fun onCreate() {
        super.onCreate()
        val cacheFactory = CacheDataSource.Factory()
            .setCache(VideoCache.get(this))                       // app-defined singleton SimpleCache
            .setUpstreamDataSourceFactory(DefaultHttpDataSource.Factory())
        val player = ExoPlayer.Builder(this)
            .setMediaSourceFactory(DefaultMediaSourceFactory(this).setDataSourceFactory(cacheFactory))
            .build()
        player.setAudioAttributes(
            AudioAttributes.Builder()
                .setUsage(C.USAGE_MEDIA)
                .setContentType(C.AUDIO_CONTENT_TYPE_MOVIE)
                .build(),
            /* handleAudioFocus= */ true,
        )
        session = MediaSession.Builder(this, player).build()
    }

    override fun onGetSession(controllerInfo: MediaSession.ControllerInfo): MediaSession? = session

    override fun onDestroy() {
        session?.run { player.release(); release() }
        session = null
        super.onDestroy()
    }
}

Points to say out loud:

• Rotation. A new Activity reconnects a MediaController and re-attaches its surface; the player and its buffer are untouched.
• Picture-in-picture. Android PiP needs API 26. Declare supportsPictureInPicture on the activity, and on API 31 and above call setAutoEnterEnabled(true) in PictureInPictureParams so swiping home enters PiP without code in onUserLeaveHint. An active MediaSession supplies play and pause controls.
• Audio focus. setAudioAttributes(attributes, true) makes ExoPlayer handle focus for you, so the app has no focus callbacks to write. Media3 guidance is to set attributes that describe your content.

Disk cache for segments (Android)

Media3 caches loaded bytes with a SimpleCache plus CacheDataSource.Factory. The docs show LeastRecentlyUsedCacheEvictor(maxBytes) for on-the-fly caching, so the cache evicts old segments at a size limit, and note that the StandaloneDatabaseProvider should be a singleton. Choose maxBytes from product needs and device storage. Keep downloads in a separate cache with NoOpCacheEvictor, as the downloading docs say a download cache should not evict media.

Preloading the next video

Media3's DefaultPreloadManager preloads items by distance from the playing item, and your app decides how much of each. The docs sample code loads 3000 ms of the next and previous items (its prose says 1000 ms for the previous one), selects tracks two items away, and prepares the source up to four items away.

// Sketch, following the Media3 docs sample. The app must keep currentPlayingIndex current.
class FeedPreloadControl(var currentPlayingIndex: Int = 0) :
    TargetPreloadStatusControl<Int, DefaultPreloadManager.PreloadStatus> {

    override fun getTargetPreloadStatus(index: Int): DefaultPreloadManager.PreloadStatus =
        when (abs(index - currentPlayingIndex)) {
            1 -> DefaultPreloadManager.PreloadStatus.specifiedRangeLoaded(3000L)
            2 -> DefaultPreloadManager.PreloadStatus.PRELOAD_STATUS_TRACKS_SELECTED
            3, 4 -> DefaultPreloadManager.PreloadStatus.PRELOAD_STATUS_SOURCE_PREPARED
            else -> DefaultPreloadManager.PreloadStatus.PRELOAD_STATUS_NOT_PRELOADED
        }
}

Build the player with buildExoPlayer() on the same DefaultPreloadManager.Builder (the service sketch above uses a plain builder for brevity). DefaultLoadControl.Builder().setPlayerTargetBufferBytes("preload", ...) caps total preload memory. On cellular or data saver, return a lighter status (or not preloaded) from this control.

On iOS there is no preload manager. You create the next AVPlayerItem early and let AVFoundation load its playlist, optionally with a small preferredForwardBufferDuration. I found no documented public API for a persistent segment cache in AVPlayer, so expect to rely on the system's HTTP cache and on your CDN.

5. Offline downloads and DRM

Android. Media3's DownloadManager runs downloads through a DownloadService, writes to a Cache, and respects Requirements such as network type. DownloadHelper selects tracks for adaptive streams, so the user can pick a quality before downloading. Play downloads through a read-only CacheDataSource.Factory, and build the item with the download request's toMediaItem so only the downloaded tracks play. Do not read the download files directly. Downloads are business-oriented work; Media3 provides its own DownloadService for it, where the architecture guide names WorkManager.

iOS. AVAssetDownloadURLSession needs a background configuration, so downloads continue while the app is suspended. If the user force-quits the app, the system cancels active downloads and does not relaunch it. Save the local file URL the delegate gives you so you can rebuild the asset on the next launch.

DRM at a high level. Widevine on Android and FairPlay on iOS keep the decryption keys in the platform, and the licence server decides who can play and for how long.

• Media3 supports Widevine "cenc" from API 19 and "cbcs" from API 25, with DASH and with HLS in fragmented MP4. FairPlay is the iOS counterpart, driven through AVContentKeySession.
• For offline playback, the client stores a persistent licence. On Android the key set ID goes on the download request and the MediaItem.DrmConfiguration. On iOS, AVContentKeySession supports persistable keys and has APIs to get an expiration date and to invalidate keys.
• Licences expire. Store the expiry, show it in the downloads list, and refresh the licence while online. Show a clear message for "licence expired, no network", not a generic player error.

6. Buffering policy, network switching and data limits

• Bandwidth caps. On Android, TrackSelectionParameters accepts constraints and applies them to every item in a playlist; setMaxVideoSizeSd() is the documented example. By default ExoPlayer limits video resolution to the display size. On iOS, AVPlayerItem has preferredPeakBitRate and a separate preferredPeakBitRateForExpensiveNetworks, plus resolution equivalents, so you can cap cellular without custom network code.
• Data saver. Lower the cap, stop preloading, and stop feed autoplay.
• Buffer size. More buffer protects against stalls but wastes data if the user leaves. ExoPlayer exposes a LoadControl for buffering policy. On iOS, preferredForwardBufferDuration sets how far ahead the player buffers, and automaticallyWaitsToMinimizeStalling (on by default) delays playback when a stall looks likely.
• Network switches. Rely on the player's retry and bitrate adaptation, show a spinner only after a short delay, and never recreate the player on a connectivity change.

// Sketch: one long-lived player; per-item limits for data saver
@MainActor final class PlaybackController {
    let player = AVPlayer()

    func configureSession() {
        // Playback category; background audio also needs the Audio, AirPlay, and Picture in Picture background mode.
        try? AVAudioSession.sharedInstance().setCategory(.playback, mode: .moviePlayback)
    }

    func play(_ url: URL, dataSaver: Bool) {
        let item = AVPlayerItem(url: url)
        if dataSaver {
            item.preferredPeakBitRateForExpensiveNetworks = 1_500_000   // example value
            item.preferredForwardBufferDuration = 5                     // seconds, example value
        }
        player.replaceCurrentItem(with: item)
        try? AVAudioSession.sharedInstance().setActive(true)
        player.play()
    }

    func startupSeconds() -> Double? {
        player.currentItem?.accessLog()?.events.last?.startupTime
    }
}

Apple suggests deferring session activation until playback begins. For PiP, AVPlayerViewController has allowsPictureInPicturePlayback and canStartPictureInPictureAutomaticallyFromInline.

7. Autoplay in the feed

Autoplay the most visible item only. Compute visible fraction per row, pick the row with the largest fraction above a threshold (for example 60 percent, an example value), and move one player to it. Start muted, and pause when the row leaves or the app backgrounds.

The costs are real. Each active decoder costs hardware and battery, so run one at a time. Autoplay spends data, so gate it on Wi-Fi or a setting. Reuse one player and swap the media item, and use the preload control to hide swap latency. Show the thumbnail until the first frame renders, or the row flashes black.

8. Resume and cross-device sync

Save the position on pause, on leaving the screen, and every 30 seconds or so while playing (example interval). Write locally first and mark the row dirty (a lazy write in offline-first terms), via an injected app-scoped CoroutineScope. Upload with business-oriented work that must survive process death (a @HiltWorker on WorkManager on Android; a persisted queue drained on foreground and reachability changes on iOS).

On open, the server returns resumeMs. If a local dirty row is newer, keep it. Use last-write-wins by timestamp, and treat a position near the end as finished.

9. Analytics and QoE

On iOS, AVPlayerItemAccessLogEvent exposes startupTime, numberOfStalls, observedBitrate, indicatedBitrate and switchBitrate. On Android, collect the equivalent from player listeners and analytics callbacks. Queue and batch events (a queued write, drained by WorkManager) and slice by network type, device class and CDN, since averages hide problem cohorts.

10. Failure modes

11. How to build a video streaming app on iOS (Swift)

The main differences: iOS gives you more built in for downloads and bitrate caps, and less for caching and preloading.

12. What changes by level

• Mid-level: play HLS with ExoPlayer or AVPlayer, a lifecycle-safe player holder, save the resume position, handle errors.
• Senior: service-owned player, PiP and audio focus, bounded segment cache, preload of the next item with data-aware limits, cellular bitrate caps, offline downloads with licence expiry, field-selected feed API, QoE metrics.
• Staff: multi-CDN failover, per-cohort tuning of startup rung and buffers from QoE data, experiments on preload depth, a cost model (CDN bytes versus engagement), a shared playback module across apps, and DRM licence policy with the backend.

13. Follow-up questions to expect

1. How would you add live streaming? (Live-edge latency, shorter segments, no long preload.)
2. How do you cut startup time? (Preload, low first rung, short segments, fewer round trips before the manifest.)
3. How do you keep data costs down for users? (Quality caps, no preload on cellular, data saver, downloads on Wi-Fi.)
4. How do you roll out a player change safely? (Feature flag, compare QoE between cohorts.)
5. How do you handle captions and audio tracks? (Track selection with preferred languages.)

Further reading (official docs)

• Android architecture recommendations
• Android data layer
• Handling audio focus (Android)
• Media3 ExoPlayer: HLS
• Media3 ExoPlayer: track selection
• Media3 ExoPlayer: network stacks and caching
• Media3 ExoPlayer: downloading media
• Media3 ExoPlayer: DRM
• Media3: preloading media
• Media3: background playback with MediaSessionService
• Picture-in-picture for video (Android)
• AVPlayerItem (Apple)
• AVAssetDownloadURLSession (Apple)
• AVContentKeySession (Apple)
• AVAudioSession (Apple)
• AVPlayerViewController (Apple)
• HTTP Live Streaming (Apple)

Keep reading

• Design an analytics SDK
  Batching, persistence, privacy, versioning.
• Design a push notification SDK
  Token lifecycle, delivery, deep links.
• Design a chat app
  Outbox, idempotent sends, WebSocket and push sync.
• Guide: Staff-level mobile system design: what the round tests
  How staff answers differ from senior ones, what interviewers can observe, and a four-week practice plan.
• Guide: Native vs cross-platform: write the decision record
  A staff-level ADR for a framework bet: criteria, the assumption that flips it, migration, parity tests, orphaned libraries, rollback, and a free template.