Release Notes

v4.0.1

v4.0.1 was released on September 29, 2022.

Compatibility changes

To improve code usability, this release changes the parameters in the following classes from the optional type to basic data types such as Bool and Int:

• In AgoraMediaSource, isAgoraSource and isLiveSource are changed from Optional to Bool.
• In AgoraRtcChannelMediaOptions, all parameter types are changed from Optional to Bool or Int.

New features

1. In-ear monitoring

This release adds support for in-ear monitoring. You can call enableInEarMonitoring to enable the in-ear monitoring function.

After successfully enabling the in-ear monitoring function, you can call setAudioFrameDelegate to register the audio observer, and the SDK triggers the onEarMonitoringAudioFrame callback to report the audio frame data. You can use your own audio effect processing module to preprocess the audio frame data of the in-ear monitoring to implement custom audio effects. Agora recommends that you choose one of the following two methods to set the audio data format of the in-ear monitoring:

• Call the setEarMonitoringAudioFrameParametersWithSampleRate method to set the audio data format of in-ear monitoring. The SDK calculates the sampling interval based on the parameters in this method and triggers the onEarMonitoringAudioFrame callback based on the sampling interval.
• Set the audio data format in the return value of the getEarMonitoringAudioParams callback. The SDK calculates the sampling interval based on the return value of the callback and triggers the onEarMonitoringAudioFrame callback based on the sampling interval.

To adjust the in-ear monitoring volume, you can call setInEarMonitoringVolume.

2. Local network connection types

To make it easier for users to know the connection type of the local network at any stage, this release adds the getNetworkType method. You can use this method to get the type of network connection in use. The available values are UNKNOWN, DISCONNECTED, LAN, WIFI, 2G, 3G, 4G, and 5G. When the local network connection type changes, the SDK triggers the networkTypeChangedToType callback to report the current network connection type.

3. Audio stream filter

This release introduces filtering audio streams based on volume. Once this function is enabled, the Agora server ranks all audio streams by volume and transports the three audio streams with the highest volumes to the receivers by default. The number of audio streams to be transported can be adjusted; contact support@agora.io to adjust this number according to your scenarios.

Agora also supports publishers in choosing whether the audio streams being published are to be filtered based on volume. Streams that are not filtered bypass this filter mechanism and are transported directly to the receivers. In scenarios with a number of publishers, enabling this function helps reduce the bandwidth and device system pressure for the receivers.

4. Spatial audio effect

This release adds the following features applicable to spatial audio effect scenarios, which can effectively enhance the user's sense-of-presence experience in virtual interactive scenarios.

• Sound insulation area: You can set a sound insulation area and sound attenuation parameter by calling setZones. When the sound source (which can be a user or the media player) and the listener belong to the inside and outside of the sound insulation area, the listener experiences an attenuation effect similar to that of the sound in the real environment when it encounters a building partition. You can also set the sound attenuation parameter for the media player and the user by calling setPlayerAttenuation and setRemoteAudioAttenuation respectively, and specify whether to use that setting to force an override of the sound attenuation parameter in setZones.
• Doppler sound: You can enable Doppler sound by setting the enable_doppler parameter in SpatialAudioParams. The receiver experiences noticeable tonal changes in the event of a high-speed relative displacement between the source source and receiver (such as in a racing game scenario).
• Headphone equalizer: You can use a preset headphone equalization effect by calling the setHeadphoneEQPreset method to improve the audio experience for users with headphones.

API changes

Added

• enableInEarMonitoring
• setEarMonitoringAudioFrameParametersWithSampleRate
• onEarMonitoringAudioFrame
• setInEarMonitoringVolume
• getEarMonitoringAudioParams
• getNetworkType
• isAudioFilterable in the AgoraRtcChannelMediaOptions
• setZones
• setPlayerAttenuation
• setRemoteAudioAttenuation
• muteRemoteAudioStream
• SpatialAudioParams
• setHeadphoneEQPreset
• AgoraHeadphoneEQPreset

Deprecated

• startEchoTest [2/3]

v4.0.0

v4.0.0 was released on September 15, 2022.

Compatibility changes

1. Integration change

This release has optimized the implementation of some features, resulting in incompatibility with v3.7.0. The following are the main features with compatibility changes:

• Multiple channel
• Media stream publishing control
• Custom video capture and rendering (Media IO)
• Warning codes

After upgrading the SDK, you need to update the code in your app according to your business scenarios. For details, see Migrate from v3.7.0 to v4.0.0.

New features

1. Multiple media tracks

This release supports one AgoraRtcEngineKit instance to collect multiple audio and video sources at the same time and publish them to the remote users by setting AgoraRtcEngineKit(Ex) and AgoraRtcChannelMediaOptions.

• After calling joinChannelByToken to join the first channel, call joinChannelExByToken multiple times to join multiple channels, and publish the specified stream to different channels through different user ID (localUid) and AgoraRtcChannelMediaOptions settings.

You can also experience the following features with the multi-channel capability:

• Publish multiple sets of audio streams to the remote users through different user IDs (uid).
• Mix multiple audio streams and publish to the remote users through a user ID (uid).

3. Agora media player

To make it easier for users to integrate the Agora SDK and reduce the SDK's package size, this release introduces the Agora media player. After calling the createMediaPlayerWithDelegate method to create a media player object, you can then call the methods in the AgoraRtcMediaPlayerProtocol class to experience a series of functions, such as playing local and online media files, preloading a media file, changing the CDN route for playing according to your network conditions, or sharing the audio and video streams being played with remote users.

4. Brand-new AI noise reduction

The SDK supports a new version of AI noise reduction (in comparison to the basic AI noise reduction in v3.7.0). The new AI noise reduction has better vocal fidelity, cleaner noise suppression, and adds a dereverberation option. To experience this feature, contact sales-us@agora.io.

5. Ultra-high audio quality

To make the audio clearer and restore more details, this release adds the AgoraVoiceBeautifierUltraHighQuality enumeration. In scenarios that mainly feature the human voice, such as chat or singing, you can call setVoiceBeautifierPreset and use this enumeration to experience ultra-high audio quality.

6. Spatial audio

This feature is in experimental status. To enable this feature, contact sales-us@agora.io. Contact Technical Support if needed.

You can set the spatial audio for the remote user as following:

• Local Cartesian Coordinate System Calculation: This solution uses the AgoraLocalSpatialAudioKit class to implement spatial audio by calculating the spatial coordinates of the remote user. You need to call updateSelfPosition and updateRemotePosition to update the spatial coordinates of the local and remote users, respectively, so that the local user can hear the spatial audio effect of the remote user.
  

You can also set the spatial audio for the media player as following:

• Local Cartesian Coordinate System Calculation: This solution uses the AgoraLocalSpatialAudioKit class to implement spatial audio. You need to call updateSelfPosition and updatePlayerPositionInfo to update the spatial coordinates of the local user and media player, respectively, so that the local user can hear the spatial audio effect of media player.
  

7. Real-time chorus

This release gives real-time chorus the following abilities:

• Two or more choruses are supported.
• Each singer is independent of each other. If one singer fails or quits the chorus, the other singers can continue to sing.
• Very low latency experience. Each singer can hear each other in real time, and the audience can also hear each singer in real time.

This release adds the AgoraAudioScenarioChorus enumeration in AgoraAudioScenario. With this enumeration, users can experience ultra-low latency in real-time chorus when the network conditions are good.

8. Extensions from the Agora extensions marketplace

In order to enhance the real-time audio and video interactive activities based on the Agora SDK, this release supports the one-stop solution for the extensions from the Agora extensions marketplace:

• Easy to integrate: The integration of modular functions can be achieved simply by calling an API, and the integration efficiency is improved by nearly 95%.
• Extensibility design: The modular and extensible SDK design style endows the Agora SDK with good extensibility, which enables developers to quickly build real-time interactive apps based on the Agora extensions marketplace ecosystem.
• Build an ecosystem: A community of real-time audio and video apps has developed that can accommodate a wide range of developers, offering a variety of extension combinations. After integrating the extensions, developers can build richer real-time interactive functions. For details, see Use an Extension.
• Become a vendor: Vendors can integrate their products with Agora SDK in the form of extensions, display and publish them in the Agora extensions marketplace, and build a real-time interactive ecosystem for developers together with Agora. For details on how to develop and publish extensions, see Become a Vendor.

9. Enhanced channel management

To meet the channel management requirements of various business scenarios, this release adds the following functions to the AgoraRtcChannelMediaOptions structure:

• Sets or switches the publishing of multiple audio sources.
• Sets or switches channel profile and user role.
• Controls audio publishing delay.

Set AgoraRtcChannelMediaOptions when calling joinChannelByToken or joinChannelExByToken to specify the publishing and subscription behavior of a media stream, for example, whether to subscribe to the audio and video streams of remote users. After joining the channel, call updateChannelWithMediaOptions to update the settings in AgoraRtcChannelMediaOptions at any time, for example, to switch the published audio sources.

11. Subscription allowlists and blocklists

This release introduces subscription allowlists and blocklists for remote audio streams. You can add a user ID that you want to subscribe to in your allowlist, or add a user ID for the streams you do not wish to see to your blocklists. You can experience this feature through the following APIs, and in scenarios that involve multiple channels, you can call the following methods in the AgoraRtcEngineKit(Ex) interface:

• setSubscribeAudioBlacklist：Set the audio subscription blocklist.
• setSubscribeAudioWhitelist：Set the audio subscription allowlist.

If a user is added in a blocklist and a allowlist at the same time, only the blocklist takes effect.

12. Set audio scenarios

To make it easier to change audio scenarios, this release adds the setAudioScenario method. For example, if you want to change the audio scenario from AgoraAudioScenarioDefault to AgoraAudioScenarioGameStreaming when you are in a channel, you can call this method.

Improvements

1. Fast channel switching

This release can achieve the same switching speed as switchChannelByToken in v3.7.0 through the leaveChannel and joinChannelByToken methods so that you don't need to take the time to call the switchChannelByToken method.

2. Voice pitch of the local user
This release adds voicePitch in AgoraRtcAudioVolumeInfo of reportAudioVolumeIndicationOfSpeakers. You can use voicePitch to get the local user's voice pitch and perform business functions such as rating for singing.