Video Stream Fallback

Introduction

The audio and video quality of interactive live video streaming or a video call deteriorates under poor network conditions. To improve the efficiency of interactive live video streaming or a video call, the setLocalPublishFallbackOption and setRemoteSubscribeFallbackOption methods are used for the SDK to automatically switch the high-video stream to low-video stream and disable the video stream under these conditions.

Implementation

Before proceeding, ensure that you implement interactive live video streaming or a video call in your project. See Start Live Interactive Video Streaming or Start a Call for details.

Refer to the following steps to set the stream fallback under poor network conditions:

1. After joining the channel, the publisher calls the enableDualStreamMode method to enable dual stream mode.

2. The publisher calls the setLocalPublishFallbackOption method and set AudioOnly (2) to enable the fallback option for the locally published media stream based on the network conditions. The SDK will disable the upstream video but enable audio only when the network conditions deteriorate and cannot support both video and audio streams.

   Agora does not recommend using this method for Media Push, because the remote CDN live user will have a noticeable lag when the locally published video stream falls back to audio only.

   When the local video stream falls back to audio only or when the audio-only stream switches back to the video stream, the SDK triggers the didLocalPublishFallbackToAudioOnly callback to report current stream state to the local publisher.

3. The subscriber in the channel calls the setRemoteSubscribeFallbackOption method to set the subscribed stream fallback under poor network conditions.

   • Set StreamLow (1) to only subscribe to the low-video stream from the publisher under poor network conditions.

   • SetsAudioOnly (2) to subscribe to the low-video stream or even audio stream from the publisher under poor network conditions.

     Once the remote media stream switches to the low stream due to poor network conditions, you can monitor the stream switch between a high and low stream in the remoteVideoStats callback. When the remotely subscribed video stream falls back to audio only or when the audio-only stream switches back to the video stream, the SDK triggers the didRemoteSubscribeFallbackToAudioOnly callback.

Sample code

//Swift
// Enable the dual-stream mode (Configuration for the local).
agoraKit.enableDualStreamMode(true)

// Configurations for the publisher. When the network condition is poor, send audio stream only. 
agoraKit.setLocalPublishFallbackOption(.audioOnly)

// Configurations for the subscriber. Try to receive low stream under poor network conditions. When the current network conditions are not sufficient for video streams, receive audio stream only. 
agoraKit.setRemoteSubscribeFallbackOption(.audioOnly)

//Objective-C
// Enable the dual-stream mode (Configuration for the local).
[agoraKit enableDualStreamMode: YES];

// Configuration for the publisher. When the network condition is poor, send audio stream only. 
[agoraKit setLocalPublishFallbackOption: AgoraStreamFallbackOptionAudioOnly];

// Configuration for the subscriber. Try to receive low stream under poor network conditions. When the current network conditions are not sufficient for video streams, receive audio stream only. 
[agoraKit setRemoteSubscribeFallbackOption: AgoraStreamFallbackOptionAudioOnly];

API reference

• enableDualStreamMode
• setLocalPublishFallbackOption
• setRemoteSubscribeFallbackOption
• didLocalPublishFallbackToAudioOnly
• didRemoteSubscribeFallbackToAudioOnly
• remoteVideoStats