Audio and Video Stream Management
Introduces methods and callbacks related to the management of audio and video streams.
enableDualStreamMode [1/2]
Enables or disables dual-stream mode on the sender side.
virtual int enableDualStreamMode(bool enabled) = 0;
- High-quality video stream: High bitrate, high resolution.
- Low-quality video stream: Low bitrate, low resolution.
After you enable dual-stream mode, you can call setRemoteVideoStreamType to choose to receive either the high-quality video stream or the low-quality video stream on the subscriber side.
- This method is applicable to all types of streams from the sender, including but not limited to video streams collected from cameras, screen sharing streams, and custom-collected video streams.
- If you need to enable dual video streams in a multi-channel scenario, you can call the enableDualStreamModeEx method.
- You can call this method either before or after joining a channel.
Parameters
- enabled
-
Whether to enable dual-stream mode.
true
: Enable dual-stream mode.false
: (Default) Disable dual-stream mode.
Returns
- 0: Success.
- < 0: Failure.
enableDualStreamMode [2/2]
Enables or disables the dual-stream mode on the sender and sets the low-quality video stream.
virtual int enableDualStreamMode(bool enabled, const SimulcastStreamConfig& streamConfig) = 0;
- High-quality video stream: High bitrate, high resolution.
- Low-quality video stream: Low bitrate, low resolution.
After you enable dual-stream mode, you can call setRemoteVideoStreamType to choose to receive either the high-quality video stream or the low-quality video stream on the subscriber side.
- This method is applicable to all types of streams from the sender, including but not limited to video streams collected from cameras, screen sharing streams, and custom-collected video streams.
- If you need to enable dual video streams in a multi-channel scenario, you can call the enableDualStreamModeEx method.
- You can call this method either before or after joining a channel.
Parameters
- enabled
-
Whether to enable dual-stream mode:
true
: Enable dual-stream mode.false
: (Default) Disable dual-stream mode.
- streamConfig
-
The configuration of the low-quality video stream. See SimulcastStreamConfig.
Returns
- 0: Success.
- < 0: Failure.
muteAllRemoteAudioStreams
Stops or resumes subscribing to the audio streams of all remote users.
virtual int muteAllRemoteAudioStreams(bool mute) = 0;
After successfully calling this method, the local user stops or resumes subscribing to the audio streams of all remote users, including all subsequent users.
- Call this method after joining a channel.
- If you do not want to subscribe the audio streams of remote users before joining a channel, you can call joinChannel [2/2] and set autoSubscribeAudio as
false
. - See recommended settings in Set the Subscribing State.
Parameters
- mute
-
Whether to stop subscribing to the audio streams of all remote users:
true
: Stop subscribing to the audio streams of all remote users.false
: (Default) Subscribe to the audio streams of all remote users by default.
Returns
- 0: Success.
- < 0: Failure.
muteAllRemoteVideoStreams
Stops or resumes subscribing to the video streams of all remote users.
virtual int muteAllRemoteVideoStreams(bool mute) = 0;
After successfully calling this method, the local user stops or resumes subscribing to the audio streams of all remote users, including all subsequent users.
- Call this method after joining a channel.
- If you do not want to subscribe the video streams of remote users before joining a channel, you can call joinChannel [2/2] and set autoSubscribeVideo as
false
.
Parameters
- mute
-
Whether to stop subscribing to the video streams of all remote users.
true
: Stop subscribing to the video streams of all remote users.false
: (Default) Subscribe to the audio streams of all remote users by default.
Returns
- 0: Success.
- < 0: Failure.
muteLocalAudioStream
Stops or resumes publishing the local audio stream.
virtual int muteLocalAudioStream(bool mute) = 0;
Parameters
- mute
-
Whether to stop publishing the local audio stream.
true
: Stop publishing the local audio stream.false
: (Default) Resumes publishing the local audio stream.
Returns
- 0: Success.
- < 0: Failure.
muteLocalVideoStream
Stops or resumes publishing the local video stream.
virtual int muteLocalVideoStream(bool mute) = 0;
A successful call of this method triggers the onUserMuteVideo callback on the remote client.
- This method executes faster than the enableLocalVideo(
false
) method, which controls the sending of the local video stream. - This method does not affect any ongoing video recording, because it does not disable the camera.
Parameters
- mute
-
Whether to stop publishing the local video stream.
true
: Stop publishing the local video stream.false
: (Default) Publish the local video stream.
Returns
- 0: Success.
- < 0: Failure.
muteRemoteAudioStream
Cancels or resumes subscribing to the specified remote user's audio stream.
virtual int muteRemoteAudioStream(uid_t uid, bool mute) = 0;
- Call this method after joining a channel.
Parameters
- uid
- The user ID of the specified user.
- mute
-
Whether to stop subscribing to the audio stream of the specified user.
true
: Unsubscribe from the specified user's audio stream.false
: (Default) Subscribes to the specified user's audio stream.
Returns
- 0: Success.
- < 0: Failure.
muteRemoteVideoStream
Cancels or resumes subscribing to the specified remote user's video stream.
virtual int muteRemoteVideoStream(uid_t userId, bool mute) = 0;
- Call this method after joining a channel.
Parameters
- userId
- The user ID of the specified user.
- mute
-
Whether to subscribe to the specified remote user's video stream.
true
: Unsubscribe from the specified user's video stream.false
: (Default) Subscribes to the specified user's video stream.
Returns
- 0: Success.
- < 0: Failure.
pauseAllChannelMediaRelay
Pauses the media stream relay to all destination channels.
virtual int pauseAllChannelMediaRelay() = 0;
After the cross-channel media stream relay starts, you can call this method to pause relaying media streams to all destination channels; after the pause, if you want to resume the relay, call resumeAllChannelMediaRelay.
After a successful method call, the SDK triggers the onChannelMediaRelayEvent callback to report whether the media stream relay is successfully paused.
Returns
- 0: Success.
- < 0: Failure.
resumeAllChannelMediaRelayEx
Resumes the media stream relay to all destination channels.
virtual int resumeAllChannelMediaRelay() = 0;
After calling the pauseAllChannelMediaRelayEx method, you can call this method to resume relaying media streams to all destination channels.
After a successful method call, the SDK triggers the onChannelMediaRelayEvent callback to report whether the media stream relay is successfully resumed.
Parameters
- connection
- The connection information. See RtcConnection.
Returns
- 0: Success.
- < 0: Failure.
sendStreamMessage
Sends data stream messages.
virtual int sendStreamMessage(int streamId,
const char* data,
size_t length) = 0;
- Up to 30 packets can be sent per second in a channel with each packet having a maximum size of 1 KB.
- Each client can send up to 6 KB of data per second.
- Each user can have up to five data streams simultaneously.
A successful method call triggers the onStreamMessage callback on the remote client, from which the remote user gets the stream message. A failed method call triggers the onStreamMessageError callback on the remote client.
- Ensure that you call createDataStream [2/2] to create a data channel before calling this method.
- In live streaming scenarios, this method only applies to hosts.
Parameters
- streamId
- The data stream ID. You can get the data stream ID by calling createDataStream [2/2].
- data
- The data to be sent.
- length
- The length of the data.
Returns
- 0: Success.
- < 0: Failure.
setDualStreamMode [1/2]
Sets dual-stream mode on the sender side.
virtual int setDualStreamMode(SIMULCAST_STREAM_MODE mode) = 0;
- Since
- v4.0.1
The SDK enables the low-quality video stream auto mode on the sender by default, which is equivalent to calling this method and setting the mode to AUTO_SIMULCAST_STREAM. If you want to modify this behavior, you can call this method and modify the mode to DISABLE_SIMULCAST_STREAM(always never send low-quality video streams) or ENABLE_SIMULCAST_STREAM (always send low-quality video streams).
- When calling this method and setting mode to DISABLE_SIMULCAST_STREAM, it has the same effect as
enableDualStreamMode [1/2](false)
. - When calling this method and setting mode to ENABLE_SIMULCAST_STREAM, it has the same effect as
enableDualStreamMode [1/2](true)
. - Both methods can be called before and after joining a channel. If they are used at the same time, the settings in the method called later shall prevail.
Parameters
- mode
- The mode in which the video stream is sent. See SIMULCAST_STREAM_MODE.
Returns
- 0: Success.
- < 0: Failure.
setDualStreamMode [2/2]
Sets dual-stream mode configuration on the sender, and sets the low-quality video stream.
virtual int setDualStreamMode(SIMULCAST_STREAM_MODE mode,
const SimulcastStreamConfig& streamConfig) = 0;
- Since
- v4.0.1
The SDK enables the low-quality video stream auto mode on the sender by default, which is equivalent to calling this method and setting the mode to AUTO_SIMULCAST_STREAM. If you want to modify this behavior, you can call this method and modify the mode to DISABLE_SIMULCAST_STREAM(always never send low-quality video streams) or ENABLE_SIMULCAST_STREAM (always send low-quality video streams).
The difference between this method and setDualStreamMode [1/2] is that this method can also configure the low-quality video stream, and the SDK sends the stream according to the configuration in streamConfig.
- When calling this method and setting mode to DISABLE_SIMULCAST_STREAM, it has the same effect as
enableDualStreamMode [1/2](false)
. - When calling this method and setting mode to ENABLE_SIMULCAST_STREAM, it has the same effect as
enableDualStreamMode [1/2](true)
. - Both methods can be called before and after joining a channel. If they are used at the same time, the settings in the method called later shall prevail.
Parameters
- mode
- The mode in which the video stream is sent. See SIMULCAST_STREAM_MODE.
- streamConfig
-
The configuration of the low-quality video stream. See SimulcastStreamConfig.
Returns
- 0: Success.
- < 0: Failure.
setRemoteDefaultVideoStreamType
Sets the default stream type of subscrption for remote video streams.
virtual int setRemoteDefaultVideoStreamType(VIDEO_STREAM_TYPE streamType) = 0;
Under limited network conditions, if the publisher has not disabled the dual-stream mode using enableDualStreamMode [2/2] (false)
, the receiver can choose to receive either the high-quality video stream or the low-quality video stream. The high-quality video stream has a higher resolution and bitrate, and the low-quality video stream has a lower resolution and bitrate.
By default, users receive the high-quality video stream. Call this method if you want to switch to the low-quality video stream. This method allows the app to adjust the corresponding video stream type based on the size of the video window to reduce the bandwidth and resources. The aspect ratio of the low-quality video stream is the same as the high-quality video stream. Once the resolution of the high-quality video stream is set, the system automatically sets the resolution, frame rate, and bitrate of the low-quality video stream.
The SDK enables the low-quality video stream auto mode on the sender by default (not actively sending low-quality video streams). The host at the receiving end can call this method to initiate a low-quality video stream stream request on the receiving end, and the sender automatically switches to the low-quality video stream mode after receiving the request.
The result of this method returns in the onApiCallExecuted callback.
- Call this method before joining a channel. Agora does not support changing the default subscribed video stream type after joining a channel.
- If you call both this method and setRemoteVideoStreamType, the SDK applies the settings in the setRemoteVideoStreamType method.
Parameters
- streamType
-
The default video-stream type. See VIDEO_STREAM_TYPE.
Returns
- 0: Success.
- < 0: Failure.
setRemoteVideoStreamType
Sets the stream type of the remote video.
virtual int setRemoteVideoStreamType(uid_t uid, VIDEO_STREAM_TYPE streamType) = 0;
Under limited network conditions, if the publisher has not disabled the dual-stream mode using enableDualStreamMode [2/2](false)
, the receiver can choose to receive either the high-quality video stream or the low-quality video stream. The high-quality video stream has a higher resolution and bitrate, and the low-quality video stream has a lower resolution and bitrate.
By default, users receive the high-quality video stream. Call this method if you want to switch to the low-quality video stream. This method allows the app to adjust the corresponding video stream type based on the size of the video window to reduce the bandwidth and resources. The aspect ratio of the low-quality video stream is the same as the high-quality video stream. Once the resolution of the high-quality video stream is set, the system automatically sets the resolution, frame rate, and bitrate of the low-quality video stream.
The SDK enables the low-quality video stream auto mode on the sender by default (not actively sending low-quality video streams). The host at the receiving end can call this method to initiate a low-quality video stream stream request on the receiving end, and the sender automatically switches to the low-quality video stream mode after receiving the request.
The result of this method returns in the onApiCallExecuted callback.
Parameters
- uid
- The user ID.
- streamType
-
The video stream type: VIDEO_STREAM_TYPE.
Returns
- 0: Success.
- < 0: Failure.
setRemoteVideoSubscriptionOptions
Options for subscribing to remote video streams.
virtual int setRemoteVideoSubscriptionOptions(uid_t uid, const VideoSubscriptionOptions &options) = 0;
When a remote user has enabled dual-stream mode, you can call this method to choose the option for subscribing to the video streams sent by the remote user.
- If you only register one IVideoFrameObserver object, the SDK subscribes to the raw video data and encoded video data by default (the effect is equivalent to setting encodedFrameOnly to
false
). - If you only register one IVideoEncodedFrameObserver object, the SDK only subscribes to the encoded video data by default (the effect is equivalent to setting encodedFrameOnly to
true
). - If you register one IVideoFrameObserver object and one IVideoEncodedFrameObserver object successively, the SDK subscribes to the encoded video data by default (the effect is equivalent to setting encodedFrameOnly to
false
). - If you call this method first with the options parameter set, and then register one IVideoFrameObserver or IVideoEncodedFrameObserver object, you need to call this method again and set the options parameter as described in the above two items to get the desired results.
- Set autoSubscribeVideo to
false
when calling joinChannel [2/2] to join a channel. - Call this method after receiving the onUserJoined callback to set the subscription options for the specified remote user's video stream.
- Call the muteRemoteVideoStream method to resume subscribing to the video stream of the specified remote user. If you set encodedFrameOnly to
true
in the previous step, the SDK triggers the onEncodedVideoFrameReceived callback locally to report the received encoded video frame information.
Parameters
- uid
- The user ID of the remote user.
- options
- The video subscription options. See VideoSubscriptionOptions.
Returns
- 0: Success.
- < 0: Failure.
setSubscribeAudioBlocklist
Set the blocklist of subscriptions for audio streams.
virtual int setSubscribeAudioBlocklist(uid_t* uidList, int uidNumber) = 0;
You can call this method to specify the audio streams of a user that you do not want to subscribe to.
- You can call this method either before or after joining a channel.
- The blocklist is not affected by the setting in muteRemoteAudioStream,muteAllRemoteAudioStreams, and autoSubscribeAudio in ChannelMediaOptions.
- Once the blocklist of subscriptions is set, it is effective even if you leave the current channel and rejoin the channel.
- If a user is added in the allowlist and blocklist at the same time, only the blocklist takes effect.
Parameters
- uidList
-
The user ID list of users that you do not want to subscribe to.
If you want to specify the audio streams of a user that you do not want to subscribe to, add the user ID in this list. If you want to remove a user from the blocklist, you need to call the setSubscribeAudioBlocklist method to update the user ID list; this means you only add the uid of users that you do not want to subscribe to in the new user ID list.
- uidNumber
- The number of users in the user ID list.
Returns
- 0: Success.
- < 0: Failure.
setSubscribeAudioAllowlist
Sets the allowlist of subscriptions for audio streams.
virtual int setSubscribeAudioAllowlist(uid_t* uidList, int uidNumber) = 0;
You can call this method to specify the audio streams of a user that you want to subscribe to.
- You can call this method either before or after joining a channel.
- The allowlist is not affected by the setting in muteRemoteAudioStream, muteAllRemoteAudioStreams and autoSubscribeAudio in ChannelMediaOptions.
- Once the allowlist of subscriptions is set, it is effective even if you leave the current channel and rejoin the channel.
- If a user is added in the allowlist and blocklist at the same time, only the blocklist takes effect.
Parameters
- uidList
-
The user ID list of users that you want to subscribe to.
If you want to specify the audio streams of a user for subscription, add the user ID in this list. If you want to remove a user from the allowlist, you need to call the setSubscribeAudioAllowlist method to update the user ID list; this means you only add the uid of users that you want to subscribe to in the new user ID list.
- uidNumber
- The number of users in the user ID list.
Returns
- 0: Success.
- < 0: Failure.
setSubscribeVideoBlocklist
Set the blocklist of subscriptions for video streams.
virtual int setSubscribeVideoBlocklist(uid_t* uidList, int uidNumber) = 0;
You can call this method to specify the video streams of a user that you do not want to subscribe to.
- You can call this method either before or after joining a channel.
- The blocklist is not affected by the setting in muteRemoteVideoStream, muteAllRemoteVideoStreams and autoSubscribeAudio in ChannelMediaOptions.
- Once the blocklist of subscriptions is set, it is effective even if you leave the current channel and rejoin the channel.
- If a user is added in the allowlist and blocklist at the same time, only the blocklist takes effect.
Parameters
- uidList
-
The user ID list of users that you do not want to subscribe to.
If you want to specify the video streams of a user that you do not want to subscribe to, add the user ID of that user in this list. If you want to remove a user from the blocklist, you need to call the setSubscribeVideoBlocklist method to update the user ID list; this means you only add the uid of users that you do not want to subscribe to in the new user ID list.
- uidNumber
- The number of users in the user ID list.
Returns
- 0: Success.
- < 0: Failure.
setSubscribeVideoAllowlist
Set the allowlist of subscriptions for video streams.
virtual int setSubscribeVideoAllowlist(uid_t* uidList, int uidNumber) = 0;
You can call this method to specify the video streams of a user that you want to subscribe to.
- You can call this method either before or after joining a channel.
- The allowlist is not affected by the setting in muteRemoteVideoStream, muteAllRemoteVideoStreams and autoSubscribeAudio in ChannelMediaOptions.
- Once the allowlist of subscriptions is set, it is effective even if you leave the current channel and rejoin the channel.
- If a user is added in the allowlist and blocklist at the same time, only the blocklist takes effect.
Parameters
- uidList
-
The user ID list of users that you want to subscribe to.
If you want to specify the video streams of a user for subscription, add the user ID of that user in this list. If you want to remove a user from the allowlist, you need to call the setSubscribeVideoAllowlist method to update the user ID list; this means you only add the uid of users that you want to subscribe to in the new user ID list.
- uidNumber
- The number of users in the user ID list.
Returns
- 0: Success.
- < 0: Failure.
startChannelMediaRelay
Starts relaying media streams across channels. This method can be used to implement scenarios such as co-host across channels.
virtual int startChannelMediaRelay(const ChannelMediaRelayConfiguration &configuration) = 0;
- If the onChannelMediaRelayStateChanged callback returns RELAY_STATE_RUNNING (2) and RELAY_OK (0), and the onChannelMediaRelayEvent callback returns RELAY_EVENT_PACKET_SENT_TO_DEST_CHANNEL (4), it means that the SDK starts relaying media streams between the source channel and the destination channel.
- If the onChannelMediaRelayStateChanged callback returnsRELAY_STATE_FAILURE (3), an exception occurs during the media stream relay.
- Call this method after joining the channel.
- This method takes effect only when you are a host in a live streaming channel.
- After a successful method call, if you want to call this method again, ensure that you call the stopChannelMediaRelay method to quit the current relay.
- The relaying media streams across channels function needs to be technical support enabled.
- We do not support string user accounts in this API.
Parameters
- configuration
- The configuration of the media stream relay. See ChannelMediaRelayConfiguration.
Returns
- 0: Success.
- < 0: Failure.
- -1: A general error occurs (no specified reason).
- -2: The parameter is invalid.
- -7: The method call was rejected. It may be because the SDK has not been initialized successfully, or the user role is not an host.
- -8: Internal state error. Probably because the user is not an audience member.
stopChannelMediaRelay
Stops the media stream relay. Once the relay stops, the host quits all the destination channels.
virtual int stopChannelMediaRelay() = 0;
After a successful method call, the SDK triggers the onChannelMediaRelayStateChanged callback. If the callback reports RELAY_STATE_IDLE (0) and RELAY_OK (0), the host successfully stops the relay.
Returns
- 0: Success.
- < 0: Failure.
onAudioSubscribeStateChanged
Occurs when the audio subscribing state changes.
virtual void onAudioSubscribeStateChanged(const char* channel, uid_t uid, STREAM_SUBSCRIBE_STATE oldState, STREAM_SUBSCRIBE_STATE newState, int elapseSinceLastState) { (void)channel; (void)uid; (void)oldState; (void)newState; (void)elapseSinceLastState; }
Parameters
- channel
- The channel name.
- uid
- The user ID of the remote user.
- oldState
- The previous subscribing status, see STREAM_SUBSCRIBE_STATE for details.
- newState
- The current subscribing status, see STREAM_SUBSCRIBE_STATE for details.
- elapseSinceLastState
- The time elapsed (ms) from the previous state to the current state.
onChannelMediaRelayEvent
Reports events during the media stream relay.
virtual void onChannelMediaRelayEvent(int code) { (void)code; }
Parameters
- code
-
The event code of channel media relay. See CHANNEL_MEDIA_RELAY_EVENT.