Agora provides spatial audio effects to give users an immersive audio experience in scenarios such as esports competitions and online conferences.
Agora provides users with local Cartesian coordinate system calculation solution:
This solution uses the ILocalSpatialAudioEngine
class to implement spatial audio effects and calculates the relative positions of the local and remote users through the SDK. You need to call updateSelfPosition
and updateRemotePosition
to update the spatial coordinates of the local and remote users, respectively, so that the local user can hear the remote user's spatial audio effects.
Agora provides media player with local Cartesian coordinate system calculation solution:
This solution calculates the relative positions of the local user and the media player through the SDK. You need to call updateSelfPosition
and updatePlayerPositionInfo
in the ILocalSpatialAudioEngine
class to update the spatial coordinates of the local user and the media player, respectively, so that the local user can hear the spatial audio effect of the media player.
This solution implements spatial audio effects through the ILocalSpatialAudioEngine
class. The API call sequence and operation steps are as follows:
Before calling other Agora APIs, call createAgoraRtcEngine
and initialize
of the IRtcEngine
class and fill in your App ID to initialize the IRtcEngine
object.
Before calling other APIs of the ILocalSpatialAudioEngine
class, call initialize
of the ILocalSpatialAudioEngine
class to initialize the ILocalSpatialAudioEngine
object.
Call setAudioProfile
to set the profile
according to your needs and the scenario
to AUDIO_SCENARIO_GAME_STREAMING
.
Call joinChannel
with the options
parameter to join the channel (using RTC Token). Set channelProfile
to CHANNEL_PROFILE_LIVE_BROADCASTING
and clientRoleType
to CLIENT_ROLE_BROADCASTER
in ChannelMediaOptions
.
Agora subscribes to the audio streams of all remote users by default. You need to call muteAllRemoteAudioStreams(true)
of IRtcEngine
to unsubscribe all remote users, otherwise the audio reception range you set in step 6 will be invalid.
Call the following method to set the audio reception range:
setMaxAudioRecvCount
to set the maximum number of audio streams that a user can receive in a specified audio reception range.setAudioRecvRange
to set the audio reception range (meters) of the local user.You need to call updateSelfPosition
and updateRemotePosition
to update the spatial coordinates of the local and the remote users successively, so that the local user can hear the remote user's spatial audio effects.
If you do not need to experience spatial audio effects, call clearRemotePositions
to delete the spatial position information of all remote users. After the deletion, the local user cannot hear all remote users.
Call release
of ILocalSpatialAudioEngine
to destroy the ILocalSpatialAudioEngine
object.
ILocalSpatialAudioEngine
object must be destroyed before calling release
of the IRtcEngine
.Call leaveChannel
and release
of the IRtcEngine
to leave the channel and destroy the IRtcEngine
object.
This solution implements spatial audio effects through the updateSelfPosition
and updatePlayerPositionInfo
in the ILocalSpatialAudioEngine
class. Taking the ILocalSpatialAudioEngine
class as an example, the operation steps are as follows:
Before calling other Agora APIs, call createAgoraRtcEngine
and initialize
of the IRtcEngine
class and fill in your App ID to initialize the IRtcEngine
object.
Before calling other APIs of the ILocalSpatialAudioEngine
class, call initialize
to initialize the ILocalSpatialAudioEngine
object.
Call setAudioProfile
to set the profile
to AUDIO_PROFILE_DEFAULT
and the scenario
to AUDIO_SCENARIO_GAME_STREAMING
.
Call joinChannel
with the options
parameter to join the channel (using RTC Token). Set channelProfile
to CHANNEL_PROFILE_LIVE_BROADCASTING
and clientRoleType
to CLIENT_ROLE_BROADCASTER
in ChannelMediaOptions
.
Agora subscribes to the audio streams of all remote users by default. You need to call muteAllRemoteAudioStreams(true)
of IRtcEngine
to unsubscribe all remote users, otherwise the audio reception range you set in step 6 will be invalid.
Call the following method to set the audio reception range:
setMaxAudioRecvCount
to set the maximum number of audio streams that a user can receive in a specified audio reception range.setAudioRecvRange
to set the audio reception range (meters) of the local user.You need to call updateSelfPosition
and updateRemotePosition
to update the spatial coordinates of the local and the remote users successively, so that the local user can hear the remote user's spatial audio effects.
Call release
of ILocalSpatialAudioEngine
to destroy the ILocalSpatialAudioEngine
object.
ILocalSpatialAudioEngine
object must be destroyed before calling release
of the IRtcEngine
.Call release
of IRtcEngine
to destroy the IRtcEngine
object.
This section shows the sample code that uses the local spherical coordinate system calculation solution to implement the spatial audio effect for users:
bool CAgoraSpatialAudioDlg::InitAgora()
{
// Create an IRtcEngine
m_rtcEngine = createAgoraRtcEngine();
RtcEngineContext context;
std::string strAppID = GET_APP_ID;
context.appId = strAppID.c_str();
context.eventHandler = &m_eventHandler;
// Initalize the IRtcEngine
int ret = m_rtcEngine->initialize(context);
// Set the channel profile
m_rtcEngine->setChannelProfile(CHANNEL_PROFILE_LIVE_BROADCASTING);
// Set the client role
m_rtcEngine->setClientRole(CLIENT_ROLE_BROADCASTER);
// Join in the channel
ChannelMediaOptions options;
options.channelProfile = CHANNEL_PROFILE_LIVE_BROADCASTING;
options.clientRoleType = CLIENT_ROLE_BROADCASTER;
m_rtcEngine->joinChannel("Your token", szChannelId.c_str(), 0, options);
// Enable spatial audio effect
m_rtcEngine->enableSpatialAudio(true);
}
void CAgoraSpatialAudioDlg::UnInitAgora()
{
if (m_rtcEngine) {
if (m_joinChannel)
// Leave the channel
m_joinChannel = !m_rtcEngine->leaveChannel();
// Destroy the IRtcEngine
m_rtcEngine->release(true);
m_rtcEngine = NULL;
}
}
// Enable spatial audio effect for the media player
void CAgoraSpatialAudioDlg::SetSpatialAudioParam()
{
POINT ptLocal = { rcLocal.left + localWidth / 2.0f,rcLocal.top + localWidth / 2.0f };
POINT ptRemote = { rcRemote.left + remoteWidth / 2.0f,rcRemote.top + remoteWidth / 2.0f };
SpatialAudioParams spatial_audio_params;
int deltaX = ptRemote.x - ptLocal.x;
int deltaY = ptLocal.y - ptRemote.y;
int tanAngle = 0;
float tanValue = 1;
if (deltaX == 0) {
if (deltaY > 0)
tanAngle = 360;
else if (deltaY < 0)
tanAngle = 180;
}else{
tanValue = fabs(deltaY) / fabs(deltaX);
tanAngle = atan(tanValue) * 180.0 / 3.1415926;
}
int spatialAngle = 0.0;
if (deltaX > 0 && deltaY > 0) {
spatialAngle = 270.0 + tanAngle;
}
else if (deltaX < 0 && deltaY > 0) {
spatialAngle = 90.0 - tanAngle;
}
else if (deltaX < 0 && deltaY < 0) {
spatialAngle = 90.0 + tanAngle;
}
else if (deltaX > 0 && deltaY < 0) {
spatialAngle = 270.0 - tanAngle;
}
CString str;
str.Format(_T("%d\n"), spatialAngle);
OutputDebugString(str);
int spatialDistance = sqrt(deltaX*deltaX + deltaY * deltaY) / distanceRate;
spatial_audio_params.speaker_distance = 0;
spatial_audio_params.speaker_azimuth = spatialAngle;
spatial_audio_params.speaker_elevation = 0;
spatial_audio_params.speaker_orientation = 0;
m_rtcEngine->setRemoteUserSpatialAudioParams(uid, spatial_audio_params);
}
}
Agora provides an open source sample project SpatialAudio on GitHub. You can download this sample project to try it out or refer to the source code.