v4.0.1 was released on September 29, 2022.
To improve code usability, this release changes the parameters in the following classes from the optional type to basic data types such as Bool
and Int
:
AgoraMediaSource
, isAgoraSource
and isLiveSource
are changed from Optional
to Bool
.AgoraRtcChannelMediaOptions
, all parameter types are changed from Optional
to Bool
or Int
.The position
parameter in the destroyCustomVideoTrack
method is changed to trackid
.
This release deletes the sourceType
parameter in enableDualStreamMode
[3/3] and enableDualStreamModeEx
, as well as the enableDualStreamMode
[2/3] method, because the SDK supports enabling dual-stream mode for various video sources captured by custom capture or SDK; you no longer need to specify the video source type.
1. In-ear monitoring
This release adds the getEarMonitoringAudioParams
callback to set the audio data format of the in-ear monitoring. You can use your own audio effect processing module to preprocess the audio frame data of the in-ear monitoring to implement custom audio effects. After calling setAudioFrameDelegate
to register the audio observer, set the audio data format in the return value of the getEarMonitoringAudioParams
callback. The SDK calculates the sampling interval based on the return value of the callback and triggers the onEarMonitoringAudioFrame
callback based on the sampling interval.
2. Local network connection types
To make it easier for users to know the connection type of the local network at any stage, this release adds the getNetworkType
method. You can use this method to get the type of network connection in use. The available values are UNKNOWN, DISCONNECTED, LAN, WIFI, 2G, 3G, 4G, and 5G. When the local network connection type changes, the SDK triggers the networkTypeChangedToType
callback to report the current network connection type.
3. Audio stream filter
This release introduces filtering audio streams based on volume. Once this function is enabled, the Agora server ranks all audio streams by volume and transports the three audio streams with the highest volumes to the receivers by default. The number of audio streams to be transported can be adjusted; contact support@agora.io to adjust this number according to your scenarios.
Agora also supports publishers in choosing whether the audio streams being published are to be filtered based on volume. Streams that are not filtered bypass this filter mechanism and are transported directly to the receivers. In scenarios with a number of publishers, enabling this function helps reduce the bandwidth and device system pressure for the receivers.
4. Dual-stream mode
This release optimizes the dual-stream mode. You can call enableDualStreamMode
and enableDualStreamModeEx
before and after joining a channel.
The implementation of subscribing to a low-quality video stream is expanded. The SDK enables the low-quality video stream auto mode on the sender by default (the SDK does not send low-quality video streams). Follow these steps to enable sending low-quality video streams:
setRemoteVideoStream
or setRemoteDefaultVideoStreamType
to initiate a low-quality video stream request.If you want to modify this default behavior, you can call setDualStreamMode
[1/2] or setDualStreamMode
[2/2] and set the mode
parameter to AgoraDisableSimulcastStream
(never send low-quality video streams) or AgoraEnableSimulcastStream
(always send low-quality video streams).
5. Spatial audio effect
This release adds the following features applicable to spatial audio effect scenarios, which can effectively enhance the user's sense-of-presence experience in virtual interactive scenarios.
setZones
. When the sound source (which can be a user or the media player) and the listener belong to the inside and outside of the sound insulation area, the listener experiences an attenuation effect similar to that of the sound in the real environment when it encounters a building partition. You can also set the sound attenuation parameter for the media player and the user by calling setPlayerAttenuation
and setRemoteAudioAttenuation
respectively, and specify whether to use that setting to force an override of the sound attenuation parameter in setZones
.enable_doppler
parameter in SpatialAudioParams
. The receiver experiences noticeable tonal changes in the event of a high-speed relative displacement between the source source and receiver (such as in a racing game scenario).setHeadphoneEQPreset
method to improve the audio experience for users with headphones.1. Video information change callback
This release optimizes the trigger logic of videoSizeChangedOfUid
, which can also be triggered and report the local video size change when startPreview
is called separately.
2. Screen sharing
To improve the consistency of screen sharing across platforms, AgoraLocalVideoStreamErrorExtensionCaptureStarted
, AgoraLocalVideoStreamErrorExtensionCaptureStoped
, and AgoraLocalVideoStreamErrorExtensionCaptureDisconnected
are removed as of this release. The state
parameter when sourceType
is AgoraVideoSourceTypeScreen
is used to indicate the state of screen sharing.
This release fixed the following issues.
setVideoEncoderConfigurationEx
in the channel to increase the resolution of the video, it occasionally failed. seek
method to specify a new position for playback, the video image occasionally remained unchanged; if you called the resume
method to resume playback, the video was sometimes played at a speed faster than the original one. Added
getEarMonitoringAudioParams
getNetworkType
isAudioFilterable
in the AgoraRtcChannelMediaOptions
setDualStreamMode
[1/2]
setDualStreamMode
[2/2]
setDualStreamModeEx
AgoraSimulcastStreamMode
setZones
setPlayerAttenuation
setRemoteAudioAttenuation
muteRemoteAudioStream
SpatialAudioParams
setHeadphoneEQPreset
AgoraHeadphoneEQPreset
Modified
enableDualStreamMode
[3/3]enableDualStreamModeEx
Deprecated
startEchoTest
[2/3]Deleted
enableDualStreamMode
[2/3] AgoraLocalVideoStreamErrorExtensionCaptureStarted
AgoraLocalVideoStreamErrorExtensionCaptureStoped
AgoraLocalVideoStreamErrorExtensionCaptureDisconnected
v4.0.0 was released on September 15, 2022.
1. Integration change
This release has optimized the implementation of some features, resulting in incompatibility with v3.7.0. The following are the main features with compatibility changes:
After upgrading the SDK, you need to update the code in your app according to your business scenarios. For details, see Migrate from v3.7.0 to v4.0.0.
1. Multiple media tracks
This release supports one AgoraRtcEngineKit
instance to collect multiple audio and video sources at the same time and publish them to the remote users by setting AgoraRtcEngineKit(Ex)
and AgoraRtcChannelMediaOptions
.
joinChannelByToken
to join the first channel, call joinChannelExByToken
multiple times to join multiple channels, and publish the specified stream to different channels through different user ID (localUid
) and AgoraRtcChannelMediaOptions
settings.This release adds createCustomVideoTrack
method to implement video custom capture. You can refer to the following steps to publish multiple custom captured video in the channel:
AgoraRtcChannelMediaOptions
, set the customVideoTrackId
parameter to the ID of the video track you want to publish, and set publishCustomVideoTrack
to true
.pushVideoFrame
, and specify customVideoTrackId
as the ID of the custom video track in step 2 in order to publish the corresponding custom video source in multiple channels.You can also experience the following features with the multi-channel capability:
uid
).uid
).uid
).2. Ultra HD resolution (Beta)
In order to improve the interactive video experience, the SDK optimizes the whole process of video capture, encoding, decoding and rendering, and now supports 4K resolution. The improved FEC (Forward Error Correction) algorithm enables adaptive switches according to the frame rate and number of video frame packets, which further reduces the video stuttering rate in 4K scenes.
Additionally, you can set the encoding resolution to 4K (3840 × 2160) and the frame rate to 60 fps when calling setVideoEncoderConfiguration
. The SDK supports automatic fallback to the appropriate resolution and frame rate if your device does not support 4K.
This feature has certain requirements with regards to device performance and network bandwidth, and the supported upstream and downstream frame rates vary on different platforms. To experience this feature, contact support@agora.io.
3. Agora media player
To make it easier for users to integrate the Agora SDK and reduce the SDK's package size, this release introduces the Agora media player. After calling the createMediaPlayerWithDelegate
method to create a media player object, you can then call the methods in the AgoraRtcMediaPlayerProtocol
class to experience a series of functions, such as playing local and online media files, preloading a media file, changing the CDN route for playing according to your network conditions, or sharing the audio and video streams being played with remote users.
4. Brand-new AI noise reduction
The SDK supports a new version of AI noise reduction (in comparison to the basic AI noise reduction in v3.7.0). The new AI noise reduction has better vocal fidelity, cleaner noise suppression, and adds a dereverberation option. To experience this feature, contact sales-us@agora.io.
5. Ultra-high audio quality
To make the audio clearer and restore more details, this release adds the AgoraVoiceBeautifierUltraHighQuality
enumeration. In scenarios that mainly feature the human voice, such as chat or singing, you can call setVoiceBeautifierPreset
and use this enumeration to experience ultra-high audio quality.
6. Spatial audio
This feature is in experimental status. To enable this feature, contact sales-us@agora.io. Contact Technical Support if needed.
You can set the spatial audio for the remote user as following:
AgoraLocalSpatialAudioKit
class to implement spatial audio by calculating the spatial coordinates of the remote user. You need to call updateSelfPosition
and updateRemotePosition
to update the spatial coordinates of the local and remote users, respectively, so that the local user can hear the spatial audio effect of the remote user.You can also set the spatial audio for the media player as following:
AgoraLocalSpatialAudioKit
class to implement spatial audio. You need to call updateSelfPosition
and updatePlayerPositionInfo
to update the spatial coordinates of the local user and media player, respectively, so that the local user can hear the spatial audio effect of media player.7. Real-time chorus
This release gives real-time chorus the following abilities:
This release adds the AgoraAudioScenarioChorus
enumeration in AgoraAudioScenario
. With this enumeration, users can experience ultra-low latency in real-time chorus when the network conditions are good.
8. Extensions from the Agora extensions marketplace
In order to enhance the real-time audio and video interactive activities based on the Agora SDK, this release supports the one-stop solution for the extensions from the Agora extensions marketplace:
9. Enhanced channel management
To meet the channel management requirements of various business scenarios, this release adds the following functions to the AgoraRtcChannelMediaOptions
structure:
Set AgoraRtcChannelMediaOptions
when calling joinChannelByToken
or joinChannelExByToken
to specify the publishing and subscription behavior of a media stream, for example, whether to publish video streams captured by cameras or screen sharing, and whether to subscribe to the audio and video streams of remote users. After joining the channel, call updateChannelWithMediaOptions
to update the settings in AgoraRtcChannelMediaOptions
at any time, for example, to switch the published audio and video sources.
10. Screen sharing
This release optimizes the screen sharing function. You can enable this function in the following ways.
Call the StartScreenCapture
method before joining a channel, and then call JoinChjoinChannelByToken
[2/4] to join a channel and set publishScreenCaptureVideo
as YES
.
Call the StartScreenCapture
method after joining a channel, and then call updateChannelWithMediaOptions
to set publishScreenCaptureVideo
as YES
.
11. Subscription allowlists and blocklists
This release introduces subscription allowlists and blocklists for remote audio and video streams. You can add a user ID that you want to subscribe to in your allowlist, or add a user ID for the streams you do not wish to see to your blocklists. You can experience this feature through the following APIs, and in scenarios that involve multiple channels, you can call the following methods in the AgoraRtcEngineKit(Ex)
interface:
setSubscribeAudioBlacklist
:Set the audio subscription blocklist.setSubscribeAudioWhitelist
:Set the audio subscription allowlist.setSubscribeVideoBlacklist
:Set the video subscription blocklist.setSubscribeVideoWhitelist
:Set the video subscription allowlist.If a user is added in a blocklist and a allowlist at the same time, only the blocklist takes effect.
12. Set audio scenarios
To make it easier to change audio scenarios, this release adds the setAudioScenario
method. For example, if you want to change the audio scenario from AgoraAudioScenarioDefault
to AgoraAudioScenarioGameStreaming
when you are in a channel, you can call this method.
13. Replace video feeds with images
This release supports replacing video feeds with images when publishing video streams. You can call the enableVideoImageSource
method to enable this function and choose your own images through the options
parameter. If you disable this function, the remote users see the video feeds that you publish.
1. Fast channel switching
This release can achieve the same switching speed as switchChannelByToken
in v3.7.0 through the leaveChannel
and joinChannelByToken
methods so that you don't need to take the time to call the switchChannelByToken
method.
2. Push external video frames
This releases supports pushing video frames in I422 format. You can call the pushExternalVideoFrame
method to push such video frames to the SDK.
3. Voice pitch of the local user
This release adds voicePitch
in AgoraRtcAudioVolumeInfo
of reportAudioVolumeIndicationOfSpeakers
. You can use voicePitch
to get the local user's voice pitch and perform business functions such as rating for singing.
4. Video preview
This release improves the implementation logic of startPreview
. You can call the startPreview
method to enable video preview at any time.
5. Video types of subscription
You can call the setRemoteDefaultVideoStreamType
method to choose the video stream type when subscribing to streams.