Audio Processing
Introduces methods and callbacks related to audio processing.
adjustAudioMixingPlayoutVolume
Adjusts the volume of audio mixing for local playback.
virtual int adjustAudioMixingPlayoutVolume(int volume) = 0;
onAudioMixingStateChanged(AUDIO_MIXING_STATE_PLAYING)
callback.Parameters
- volume
- The volume of audio mixing for local playback. The value ranges between 0 and 100 (default). 100 represents the original volume.
Returns
- 0: Success.
- < 0: Failure.
adjustAudioMixingPublishVolume
Adjusts the volume of audio mixing for publishing.
virtual int adjustAudioMixingPublishVolume(int volume) = 0;
This method adjusts the audio mixing volume on the remote clients.
Call this method after calling startAudioMixing [2/2] and receiving the onAudioMixingStateChanged(AUDIO_MIXING_STATE_PLAYING)
callback.
Parameters
- volume
- The volume of audio mixing for local playback. The value ranges between 0 and 100 (default). 100 represents the original volume.
Returns
- 0: Success.
- < 0: Failure.
adjustAudioMixingVolume
Adjusts the volume during audio mixing.
virtual int adjustAudioMixingVolume(int volume) = 0;
This method adjusts the audio mixing volume on both the local client and remote clients.
- Call this method after the startAudioMixing [2/2] method.
- This method does not affect the playback volume you set through the playEffect method.
Parameters
- volume
- Audio mixing volume. The value ranges between 0 and 100. The default value is 100, which means the original volume.
Returns
- 0: Success.
- < 0: Failure.
adjustCustomAudioPublishVolume
Adjusts the volume of the custom external audio source when it is published in the channel.
virtual int adjustCustomAudioPublishVolume(int32_t sourceId, int volume) = 0;
If you want to change the volume of the audio to be published, you need to call this method again.
Parameters
- sourceId
- The ID of external audio source. If you want to publish a custom external audio source, set this parameter to the ID of the corresponding custom audio track you want to publish.
- volume
- The volume of the audio source. The value can range from 0 to 100. 0 means mute; 100 means the original volume.
Returns
- 0: Success.
- < 0: Failure.
adjustLoopbackSignalVolume
Adjusts the volume of the signal captured by the sound card.
virtual int adjustLoopbackSignalVolume(int volume) = 0;
After calling enableLoopbackRecording to enable loopback audio capturing, you can call this method to adjust the volume of the signal captured by the sound card.
Parameters
- volume
- Audio mixing volume. The value ranges between 0 and 100. The default value is 100, which means the original volume.
Returns
- 0: Success.
- < 0: Failure.
adjustPlaybackSignalVolume
Adjusts the playback signal volume of all remote users.
virtual int adjustPlaybackSignalVolume(int volume) = 0;
- This method adjusts the playback volume that is the mixed volume of all remote users.
- You can call this method either before or after joining a channel.
Parameters
- volume
-
The volume of the user. The value range is [0,400].
- 0: Mute.
- 100: (Default) The original volume.
- 400: Four times the original volume (amplifying the audio signals by four times).
Returns
- 0: Success.
- < 0: Failure.
adjustRecordingSignalVolume
Adjusts the capturing signal volume.
virtual int adjustRecordingSignalVolume(int volume) = 0;
You can call this method either before or after joining a channel.
Parameters
- volume
-
The volume of the user. The value range is [0,400].
- 0: Mute.
- 100: (Default) The original volume.
- 400: Four times the original volume (amplifying the audio signals by four times).
Returns
- 0: Success.
- < 0: Failure.
adjustUserPlaybackSignalVolume
Adjusts the playback signal volume of a specified remote user.
virtual int adjustUserPlaybackSignalVolume(unsigned int uid, int volume) = 0;
You can call this method to adjust the playback volume of a specified remote user. To adjust the playback volume of different remote users, call the method as many times, once for each remote user.
- Call this method after joining a channel.
- The playback volume here refers to the mixed volume of a specified remote user.
Parameters
- uid
- The user ID of the remote user.
- volume
- Audio mixing volume. The value ranges between 0 and 100. The default value is 100, which means the original volume.
Returns
- 0: Success.
- < 0: Failure.
configRhythmPlayer
Configures the virtual metronome.
virtual int configRhythmPlayer(const AgoraRhythmPlayerConfig& config) = 0;
After calling startRhythmPlayer, you can call this method to reconfigure the virtual metronome.
After successfully calling this method, the SDK triggers the onRhythmPlayerStateChanged callback locally to report the status of the virtual metronome.
- This method is for Android and iOS only.
- After enabling the virtual metronome, the SDK plays the specified audio effect file from the beginning, and controls the playback duration of each file according to beatsPerMinute you set in AgoraRhythmPlayerConfig. For example, if you set beatsPerMinute as
60
, the SDK plays one beat every second. If the file duration exceeds the beat duration, the SDK only plays the audio within the beat duration. - By default, the sound of the virtual metronome is published in the channel. If you do not want the sound to be heard by the remote users, you can set publishRhythmPlayerTrack in ChannelMediaOptions as
false
.
Parameters
- config
- The metronome configuration. See AgoraRhythmPlayerConfig.
Returns
- 0: Success.
- < 0: Failure.
disableAudio
Disables the audio module.
virtual int disableAudio() = 0;
- This method disables the internal engine and can be called anytime after initialization. It is still valid after one leaves channel.
- This method resets the internal engine and takes some time to take effect. Agora recommends using the following API methods to control the audio modules separately:
- enableLocalAudio: Whether to enable the microphone to create the local audio stream.
- muteLocalAudioStream: Whether to publish the local audio stream.
- muteRemoteAudioStream: Whether to subscribe and play the remote audio stream.
- muteAllRemoteAudioStreams: Whether to subscribe to and play all remote audio streams.
Returns
- 0: Success.
- < 0: Failure.
disableAudioSpectrumMonitor
Disables audio spectrum monitoring.
virtual int disableAudioSpectrumMonitor() = 0;
After calling enableAudioSpectrumMonitor, if you want to disable audio spectrum monitoring, you can call this method.
You can call this method either before or after joining a channel.
Returns
- 0: Success.
- < 0: Failure.
enableAudio
Enables the audio module.
virtual int enableAudio() = 0;
The audio mode is enabled by default.
- This method enables the internal engine and can be called anytime after initialization. It is still valid after one leaves channel.
- This method enables the audio module and takes some time to take effect. Agora recommends using the following API methods to control the audio module separately:
- enableLocalAudio: Whether to enable the microphone to create the local audio stream.
- muteLocalAudioStream: Whether to publish the local audio stream.
- muteRemoteAudioStream: Whether to subscribe and play the remote audio stream.
- muteAllRemoteAudioStreams: Whether to subscribe to and play all remote audio streams.
Returns
- 0: Success.
- < 0: Failure.
enableAudioSpectrumMonitor
Turns on audio spectrum monitoring.
virtual int enableAudioSpectrumMonitor(int intervalInMS = 100) = 0;
If you want to obtain the audio spectrum data of local or remote users, you can register the audio spectrum observer and enable audio spectrum monitoring.
You can call this method either before or after joining a channel.
Parameters
- intervalInMS
-
The interval (in milliseconds) at which the SDK triggers the onLocalAudioSpectrum and onRemoteAudioSpectrum callbacks. The default value is 100. Do not set this parameter to less than 10 milliseconds, otherwise the calling of this method fails.
Returns
- 0: Success.
- < 0: Failure.
- -2: Invalid parameters.
enableAudioVolumeIndication
Enables the reporting of users' volume indication.
virtual int enableAudioVolumeIndication(int interval, int smooth, bool reportVad) = 0;
This method enables the SDK to regularly report the volume information of the local user who sends a stream and remote users (up to three) whose instantaneous volumes are the highest to the app. Once you call this method and users send streams in the channel, the SDK triggers the onAudioVolumeIndication callback at the time interval set in this method.
Parameters
- interval
- Sets the time interval between two consecutive volume indications:
- ≤ 0: Disables the volume indication.
- > 0: Time interval (ms) between two consecutive volume indications. You need to set this parameter to an integer multiple of 200. If the value is lower than 200, the SDK automatically adjusts the value to 200.
- smooth
- The smoothing factor sets the sensitivity of the audio volume indicator. The value ranges between 0 and 10. The recommended value is 3. The greater the value, the more sensitive the indicator.
- reportVad
-
true
: Enable the voice activity detection of the local user. Once it is enabled,the vad parameter of the onAudioVolumeIndication callback reports the voice activity status of the local user.false
: (Default) Disable the voice activity detection of the local user. Once it is disabled, the vad parameter of the onAudioVolumeIndication callback does not report the voice activity status of the local user, except for the scenario where the engine automatically detects the voice activity of the local user.
Returns
- 0: Success.
- < 0: Failure.
enableInEarMonitoring
Enables in-ear monitoring.
virtual int enableInEarMonitoring(bool enabled, int includeAudioFilters) = 0;
This method enables or disables in-ear monitoring.
- Users must use earphones (wired or Bluetooth) to hear the in-ear monitoring effect.
- You can call this method either before or after joining a channel.
Parameters
- enabled
- Enables or disables in-ear monitoring.
true
: Enables in-ear monitoring.false
: (Default) Disables in-ear monitoring.
- includeAudioFilters
- The audio filter of in-ear monitoring: See EAR_MONITORING_FILTER_TYPE.
Returns
- 0: Success.
- < 0: Failure.
enableLocalAudio
Enables/Disables the local audio capture.
virtual int enableLocalAudio(bool enabled) = 0;
The audio function is enabled by default. This method disables or re-enables the local audio function to stop or restart local audio capturing.
This method does not affect receiving or playing the remote audio streams, and enableLocalAudio(false)
is applicable to scenarios where the user wants to receive remote audio streams without sending any audio stream to other users in the channel.
- This method is different from the muteLocalAudioStream method:
- enableLocalAudio: Disables/Re-enables the local audio capturing and processing. If you disable or re-enable local audio capturing using the enableLocalAudio method, the local user might hear a pause in the remote audio playback.
- muteLocalAudioStream: Sends/Stops sending the local audio streams.
- You can call this method either before or after joining a channel. Calling it before joining a channel only sets the device state, and it takes effect immediately after you join the channel.
Parameters
- enabled
-
true
: (Default) Re-enable the local audio function, that is, to start the local audio capturing device (for example, the microphone).false
: Disable the local audio function, that is, to stop local audio capturing.
Returns
- 0: Success.
- < 0: Failure.
enableLoopbackRecording
Enables loopback audio capture.
virtual int enableLoopbackRecording(bool enabled, const char* deviceName = NULL) = 0;
If you enable loopback audio capture, the output of the sound card is mixed into the audio stream sent to the other end.
- Applies to the macOS and Windows platforms only.
- macOS does not support loopback audio capture of the default sound card. If you need to use this method, use a virtual sound card and pass its name to the deviceName parameter. Agora recommends that you use Soundflower for loopback audio capture.
- You can call this method either before or after joining a channel.
Parameters
- enabled
- Whether to enable loopback audio capture.
true
: Enable loopback audio capture.false
: (Default) Disable loopback audio capture.
- deviceName
-
- macOS: The device name of the virtual sound card. The default is set to null, which means the SDK uses Soundflower for loopback audio capture.
- Windows: The device name of the sound card. The default is set to null, which means the SDK uses the sound card of your device for loopback audio capture.
Returns
- 0: Success.
- < 0: Failure.
getAudioMixingCurrentPosition
Retrieves the playback position (ms) of the music file.
virtual int getAudioMixingCurrentPosition() = 0;
Retrieves the playback position (ms) of the audio.
- You need to call this method after calling startAudioMixing [2/2] and receiving the
onAudioMixingStateChanged(AUDIO_MIXING_STATE_PLAYING)
callback. - If you need to call getAudioMixingCurrentPosition multiple times, ensure that the time interval between calling this method is more than 500 ms.
Returns
- ≥ 0: The current playback position (ms) of the audio mixing, if this method call succeeds. 0 represents that the current music file does not start playing.
- < 0: Failure.
getAudioMixingDuration
Retrieves the duration (ms) of the music file.
virtual int getAudioMixingDuration() = 0;
Retrieves the total duration (ms) of the audio file.
You need to call this method after calling startAudioMixing [2/2] and receiving the onAudioMixingStateChanged(AUDIO_MIXING_STATE_PLAYING)
callback.
Returns
- ≥ 0: The audio mixing duration, if this method call succeeds.
- < 0: Failure.
getAudioMixingPlayoutVolume
Retrieves the audio mixing volume for local playback.
virtual int getAudioMixingPlayoutVolume() = 0;
This method retrieves the audio mixing volume for local playback. You can use it to troubleshoot audio volume related issues.
You need to call this method after calling startAudioMixing [2/2] and receiving the onAudioMixingStateChanged(AUDIO_MIXING_STATE_PLAYING)
callback.
Returns
- The audio mixing volume, if this method call succeeds. The value range is [0,100].
- < 0: Failure.
getAudioMixingPublishVolume
Retrieves the audio mixing volume for publishing.
virtual int getAudioMixingPublishVolume() = 0;
This method helps to troubleshoot audio volume‑related issues.
onAudioMixingStateChanged(AUDIO_MIXING_STATE_PLAYING)
callback.Returns
- The audio mixing volume, if this method call succeeds. The value range is [0,100].
- < 0: Failure.
getAudioTrackCount
Gets the index of audio tracks of the current music file.
virtual int getAudioTrackCount() = 0;
- You need to call this method after calling startAudioMixing [2/2] and receiving the
onAudioMixingStateChanged(AUDIO_MIXING_STATE_PLAYING)
callback.
Returns
- The SDK returns the index of the audio tracks if the method call succeeds.
- < 0: Failure.
muteRecordingSignal
Whether to mute the recording signal.
virtual int muteRecordingSignal(bool mute) = 0;
Parameters
- mute
-
true
: Mute the recording signal.false
: (Default) Do not mute the recording signal.
Returns
- 0: Success.
- < 0: Failure.
pauseAudioMixing
Pauses playing the music file.
virtual int pauseAudioMixing() = 0;
Call this method after joining a channel.
Returns
- 0: Success.
- < 0: Failure.
pullAudioFrame
Pulls the remote audio data.
virtual int pullAudioFrame(IAudioFrameObserver::AudioFrame* frame) = 0;
Before calling this method, you need to call setExternalAudioSink to notify the app to enable and set the external rendering.
After a successful method call, the app pulls the decoded and mixed audio data for playback.
- This method only supports pulling data from custom audio source. If you need to pull the data captured by the SDK, do not call this method.
- Call this method after joining a channel.
- Once you enable the external audio sink, the app will not retrieve any audio data from the onPlaybackAudioFrame callback.
- The difference between this method and the onPlaybackAudioFrame callback is as follows:
- The SDK sends the audio data to the app through the onPlaybackAudioFrame callback. Any delay in processing the audio frames may result in audio jitter.
- After a successful method call, the app automatically pulls the audio data from the SDK. After setting the audio data parameters, the SDK adjusts the frame buffer and avoids problems caused by jitter in the external audio playback.
Parameters
- frame
- Pointers to AudioFrame.
Returns
- 0: Success.
- < 0: Failure.
pushAudioFrame
Pushes the external audio frame.
virtual int pushAudioFrame(MEDIA_SOURCE_TYPE type, IAudioFrameObserver::AudioFrame* frame, bool wrap = false, int sourceId = 0) = 0;
Parameters
- type
- The type of the audio recording device. See MEDIA_SOURCE_TYPE.
- frame
- The external audio frame. See AudioFrame.
- wrap
- Whether to use the placeholder. Agora recommends using the default value.
true
: Use the placeholder.false
: (Default) Do not use the placeholder.
- sourceId
- The ID of external audio source. If you want to publish a custom external audio source, set this parameter to the ID of the corresponding custom audio track you want to publish.
Returns
- 0: Success.
- < 0: Failure.
resumeAudioMixing
Resumes playing and mixing the music file.
virtual int resumeAudioMixing() = 0;
This method resumes playing and mixing the music file. Call this method when you are in a channel.
Returns
- 0: Success.
- < 0: Failure.
selectAudioTrack
Selects the audio track used during playback.
virtual int selectAudioTrack(int index) = 0;
After getting the track index of the audio file, you can call this method to specify any track to play. For example, if different tracks of a multi-track file store songs in different languages, you can call this method to set the playback language.
- For the supported formats of audio files, see https://docs.agora.io/en/faq/audio_format.
- You need to call this method after calling startAudioMixing [2/2] and receiving the
onAudioMixingStateChanged(AUDIO_MIXING_STATE_PLAYING)
callback.
Parameters
- index
- The audio track you want to specify. The value range is [0, getAudioTrackCount()].
Returns
- 0: Success.
- < 0: Failure.
setAdvancedAudioOptions
Sets audio advanced options.
virtual int setAdvancedAudioOptions(media::base::AdvancedAudioOptions &options) = 0;
If you have advanced audio processing requirements, such as capturing and sending stereo audio, you can call this method to set advanced audio options.
- This method applies to Android and iOS only.
- Call this method after calling joinChannel [2/2], enableAudio and enableLocalAudio.
Parameters
- options
- The advanced options for audio. See AdvancedAudioOptions.
Returns
- 0: Success.
- < 0: Failure.
setAudioMixingDualMonoMode
Sets the channel mode of the current audio file.
virtual int setAudioMixingDualMonoMode(media::AUDIO_MIXING_DUAL_MONO_MODE mode) = 0;
In a stereo music file, the left and right channels can store different audio data. According to your needs, you can set the channel mode to original mode, left channel mode, right channel mode, or mixed channel mode. For example, in the KTV scenario, the left channel of the music file stores the musical accompaniment, and the right channel stores the singing voice. If you only need to listen to the accompaniment, call this method to set the channel mode of the music file to left channel mode; if you need to listen to the accompaniment and the singing voice at the same time, call this method to set the channel mode to mixed channel mode.
- Call this method after calling startAudioMixing [2/2] and receiving the
onAudioMixingStateChanged (AUDIO_MIXING_STATE_PLAYING)
callback. - This method only applies to stereo audio files.
Parameters
- mode
- The channel mode. See AUDIO_MIXING_DUAL_MONO_MODE.
Returns
- 0: Success.
- < 0: Failure.
setAudioMixingPitch
Sets the pitch of the local music file.
virtual int setAudioMixingPitch(int pitch) = 0;
When a local music file is mixed with a local human voice, call this method to set the pitch of the local music file only.
onAudioMixingStateChanged(AUDIO_MIXING_STATE_PLAYING)
callback.Parameters
- pitch
- Sets the pitch of the local music file by the chromatic scale. The default value is 0, which means keeping the original pitch. The value ranges from -12 to 12, and the pitch value between consecutive values is a chromatic value. The greater the absolute value of this parameter, the higher or lower the pitch of the local music file.
Returns
- 0: Success.
- < 0: Failure.
setAudioMixingPosition
Sets the audio mixing position.
virtual int setAudioMixingPosition(int pos /*in ms*/) = 0;
Call this method to set the playback position of the music file to a different starting position, rather than playing the file from the beginning.
onAudioMixingStateChanged(AUDIO_MIXING_STATE_PLAYING)
callback.Parameters
- pos
- Integer. The playback position (ms).
Returns
- 0: Success.
- < 0: Failure.
setAudioProfile [1/2]
Sets the audio profile and audio scenario.
virtual int setAudioProfile(AUDIO_PROFILE_TYPE profile, AUDIO_SCENARIO_TYPE scenario) = 0;
- Deprecated:
- This method is deprecated. If you need to set the audio profile, use setAudioProfile [2/2]; if you need to set the audio scenario, use setAudioScenario.
- You can call this method either before or after joining a channel.
- In scenarios requiring high-quality audio, such as online music tutoring, Agora recommends you set profile as
AUDIO_PROFILE_MUSIC_HIGH_QUALITY(4)
and scenario asAUDIO_SCENARIO_GAME_STREAMING(3)
.
Parameters
- profile
-
The audio profile, including the sampling rate, bitrate, encoding mode, and the number of channels. See AUDIO_PROFILE_TYPE.
- scenario
- The audio scenarios. See AUDIO_SCENARIO_TYPE. Under different audio scenarios, the device uses different volume types.
Returns
- 0: Success.
- < 0: Failure.
setAudioProfile [2/2]
Sets the audio parameters and application scenarios.
virtual int setAudioProfile(AUDIO_PROFILE_TYPE profile) = 0;
- You can call this method either before or after joining a channel.
- In scenarios requiring high-quality audio, such as online music tutoring, Agora recommends you set
profile
asAUDIO_PROFILE_MUSIC_HIGH_QUALITY (4)
. - If you want to set the audio scenario, call initialize and set audioScenario in the RtcEngineContext struct.
Parameters
- profile
-
The audio profile, including the sampling rate, bitrate, encoding mode, and the number of channels. See AUDIO_PROFILE_TYPE.
Returns
- 0: Success.
- < 0: Failure.
setAudioScenario
Sets audio scenarios.
virtual int setAudioScenario(AUDIO_SCENARIO_TYPE scenario) = 0;
Parameters
- scenario
- The audio scenarios. See AUDIO_SCENARIO_TYPE. Under different audio scenarios, the device uses different volume types.
Returns
- 0: Success.
- < 0: Failure.
setDefaultAudioRouteToSpeakerphone
Sets the default audio playback route.
virtual int setDefaultAudioRouteToSpeakerphone(bool defaultToSpeaker) = 0;
- This method applies to Android and iOS only.
- Ensure that you call this method before joining a channel. If you need to change the audio route after joining a channel, call setEnableSpeakerphone.
Most mobile phones have two audio routes: an earpiece at the top, and a speakerphone at the bottom. The earpiece plays at a lower volume, and the speakerphone at a higher volume. When setting the default audio route, you determine whether audio playback comes through the earpiece or speakerphone when no external audio device is connected.
- Voice call: Earpiece.
- Audio broadcast: Speakerphone.
- Video call: Speakerphone.
- Video broadcast: Speakerphone.
You can call this method to change the default audio route. After a successful method call, the SDK triggers the onAudioRoutingChanged callback.
The system audio route changes when an external audio device, such as a headphone or a Bluetooth audio device, is connected. See Principles for Changing the Audio Route.
Parameters
- defaultToSpeaker
- Whether to set the speakerphone as the default audio route:
true
: Set the speakerphone as the default audio route.false
: Set the earpiece as the default audio route.
Returns
- 0: Success.
- < 0: Failure.
setEarMonitoringAudioFrameParameters
Sets the format of the in-ear monitoring raw audio data.
virtual int setEarMonitoringAudioFrameParameters(int sampleRate, int channel,
RAW_AUDIO_FRAME_OP_MODE_TYPE mode,
int samplesPerCall) = 0;
This method is used to set the in-ear monitoring audio data format reported by the onEarMonitoringAudioFrame callback.
- Before calling this method, you need to call enableInEarMonitoring, and set includeAudioFilters to EAR_MONITORING_FILTER_BUILT_IN_AUDIO_FILTERS or EAR_MONITORING_FILTER_NOISE_SUPPRESSION.
- The SDK calculates the sampling interval based on the samplesPerCall, sampleRate and channel parameters set in this method.Sample interval = samplePerCall/(sampleRate × channel). Ensure that the sample interval ≥ 0.01 (s). The SDK triggers the onEarMonitoringAudioFrame callback according to the sampling interval.
Parameters
- sampleRate
- The sample rate of the audio data reported in the onEarMonitoringAudioFrame callback, which can be set as 8,000, 16,000, 32,000, 44,100, or 48,000 Hz.
- channel
-
The number of audio channels reported in the onEarMonitoringAudioFrame callback.
- 1: Mono.
- 2: Stereo.
- mode
-
The use mode of the audio frame. See RAW_AUDIO_FRAME_OP_MODE_TYPE.
- samplesPerCall
- The number of data samples reported in the onEarMonitoringAudioFrame callback, such as 1,024 for the Media Push.
Returns
- 0: Success.
- < 0: Failure.
setExternalAudioSink
Sets the external audio sink.
virtual int setExternalAudioSink(bool enabled, int sampleRate, int channels) = 0;
This method applies to scenarios where you want to use external audio data for playback. After you set the external audio sink, you can call pullAudioFrame to pull remote audio frames. The app can process the remote audio and play it with the audio effects that you want.
Parameters
- enabled
-
Whether to enable or disable the external audio sink:
true
: Enables the external audio sink.false
: (Default) Disables the external audio sink.
- sampleRate
-
The sample rate (Hz) of the external audio sink, which can be set as 16000, 32000, 44100, or 48000.
- channels
- The number of audio channels of the external audio sink:
- 1: Mono.
- 2: Stereo.
Returns
- 0: Success.
- < 0: Failure.
setExternalAudioSource
Sets the external captured audio parameters and chooses whether to publish the audio to the remote user.
virtual int setExternalAudioSource(bool enabled,
int sampleRate,
int channels,
int sourceNumber,
bool localPlayback = false,
bool publish = true) = 0;
Parameters
- enabled
-
Whether to enable the external audio source:
true
: Enable the external audio source.false
: (Default) Disable the external audio source.
- sampleRate
- The sample rate (Hz) of the external audio source, which can be set as
8000
,16000
,32000
,44100
, or48000
. - channels
- The number of channels of the external audio source, which can be set as
1
(Mono) or2
(Stereo). - sourceNumber
- The number of external audio sources. The value of this parameter should be larger than 0. The SDK creates a corresponding number of custom audio tracks based on this parameter value and names the audio tracks starting from 0. In ChannelMediaOptions, you can set publishCustomAudioSourceId to the audio track ID you want to publish.
- localPlayback
-
Whether to play the external audio source:
true
: Play the external audio source.false
: (Default) Do not play the external source.
- publish
-
Whether to publish audio to the remote users:
true
: (Default) Publish audio to the remote users.false
: Do not publish audio to the remote users.
Returns
- 0: Success.
- < 0: Failure.
setInEarMonitoringVolume
Sets the volume of the in-ear monitor.
virtual int setInEarMonitoringVolume(int volume) = 0;
- Users must use wired earphones to hear their own voices.
- You can call this method either before or after joining a channel.
Parameters
- volume
- The volume of the in-ear monitor. The value ranges between 0 and 100. The default value is 100.
Returns
- 0: Success.
- < 0: Failure.
startAudioMixing [1/2]
Starts playing the music file.
virtual int startAudioMixing(const char* filePath,
bool loopback,
bool replace,
int cycle) = 0;
- Deprecated:
- Use startAudioMixing [2/2] instead.
This method mixes the specified local or online audio file with the audio from the microphone, or replaces the microphone's audio with the specified local or remote audio file. A successful method call triggers the onAudioMixingStateChanged (AUDIO_MIXING_STATE_PLAYING) callback. When the audio mixing file playback finishes, the SDK triggers the onAudioMixingStateChanged (AUDIO_MIXING_STATE_STOPPED) callback on the local client.
- You can call this method either before or after joining a channel. If you need to call startAudioMixing [1/2] multiple times, ensure that the time interval between calling this method is more than 500 ms.
- If the local music file does not exist, the SDK does not support the file format, or the the SDK cannot access the music file URL, the SDK reports the warn code 701.
- On Android, there are following considerations:
- To use this method, ensure that the Android device is v4.2 or later, and the API version is v16 or later.
- If you need to play an online music file, Agora does not recommend using the redirected URL address. Some Android devices may fail to open a redirected URL address.
- If you call this method on an emulator, ensure that the music file is in the
/sdcard/
directory and the format is MP3.
Parameters
- filePath
-
The absolute path or URL address (including the suffixes of the filename) of the audio effect file. For example,
C:\music\audio.mp4
. Supported audio formats include MP3, AAC, M4A, MP4, WAV, and 3GP. See supported audio formats. - loopback
-
Whether to play music files only on the local client:
true
: Only play music files on the local client so that only the local user can hear the music.false
: Publish music files to remote clients so that both the local user and remote users can hear the music.
- cycle
-
The number of times the music file plays.
- ≥ 0: The number of playback times. For example, 0 means that the SDK does not play the music file while 1 means that the SDK plays once.
- -1: Play the audio file in an infinite loop.
Returns
- 0: Success.
- < 0: Failure.
startAudioMixing [2/2]
Starts playing the music file.
virtual int startAudioMixing(const char* filePath, bool loopback, int cycle, int startPos) = 0;
This method mixes the specified local or online audio file with the audio from the microphone, or replaces the microphone's audio with the specified local or remote audio file. A successful method call triggers the onAudioMixingStateChanged(AUDIO_MIXING_STATE_PLAYING)
callback. When the audio mixing file playback finishes, the SDK triggers the onAudioMixingStateChanged(AUDIO_MIXING_STATE_STOPPED)
callback on the local client.
- You can call this method either before or after joining a channel. If you need to call startAudioMixing [2/2] multiple times, ensure that the time interval between calling this method is more than 500 ms.
- If the local music file does not exist, the SDK does not support the file format, or the the SDK cannot access the music file URL, the SDK reports the warn code 701.
- For the audio file formats supported by this method, see What formats of audio files the Agora RTC SDK support.
- On Android, there are following considerations:
- To use this method, ensure that the Android device is v4.2 or later, and the API version is v16 or later.
- If you need to play an online music file, Agora does not recommend using the redirected URL address. Some Android devices may fail to open a redirected URL address.
- If you call this method on an emulator, ensure that the music file is in the
/sdcard/
directory and the format is MP3.
Parameters
- filePath
- File path:
- Android: The file path, which needs to be accurate to the file name and suffix. Agora supports using a URI address, an absolute path, or a path that starts with
/assets/
. You might encounter permission issues if you use an absolute path to access a local file, so Agora recommends using a URI address instead. For example:content://com.android.providers.media.documents/document/audio%3A14441
. - Windows: The absolute path or URL address (including the suffixes of the filename) of the audio effect file. For example:
C:\music\audio.mp4
.
- Android: The file path, which needs to be accurate to the file name and suffix. Agora supports using a URI address, an absolute path, or a path that starts with
- loopback
-
Whether to only play music files on the local client:
true
: Only play music files on the local client so that only the local user can hear the music.false
: Publish music files to remote clients so that both the local user and remote users can hear the music.
- cycle
-
The number of times the music file plays.
- ≥ 0: The number of playback times. For example, 0 means that the SDK does not play the music file while 1 means that the SDK plays once.
- -1: Play the audio file in an infinite loop.
- startPos
- The playback position (ms) of the music file.
Returns
- 0: Success.
- < 0: Failure.
startAudioRecording
Starts the audio recording on the client.
virtual int startAudioRecording(const AudioFileRecordingConfig& config) = 0;
- WAV: High-fidelity files with typically larger file sizes. For example, the size of a WAV file with a sample rate of 32,000 Hz and a recording duration of 10 minutes is around 73 MB.
- AAC: Low-fidelity files with typically smaller file sizes. For example, if the sample rate is 32,000 Hz and the recording quality is AUDIO_RECORDING_QUALITY_MEDIUM, the file size for a 10-minute recording is approximately 2 MB.
Once the user leaves the channel, the recording automatically stops.
Parameters
- config
- Recording configuration. See AudioRecordingConfiguration.
Returns
- 0: Success.
- < 0: Failure.
startRhythmPlayer
Enables the virtual metronome.
virtual int startRhythmPlayer(const char* sound1, const char* sound2, const AgoraRhythmPlayerConfig& config) = 0;
In music education, physical education and other scenarios, teachers usually need to use a metronome so that students can practice with the correct beat. The meter is composed of a downbeat and upbeats. The first beat of each measure is called a downbeat, and the rest are called upbeats.
In this method, you need to set the file path of the upbeat and downbeat, the number of beats per measure, the beat speed, and whether to send the sound of the metronome to remote users.
After successfully calling this method, the SDK triggers the onRhythmPlayerStateChanged callback locally to report the status of the virtual metronome.
- This method is for Android and iOS only.
- After enabling the virtual metronome, the SDK plays the specified audio effect file from the beginning, and controls the playback duration of each file according to beatsPerMinute you set in AgoraRhythmPlayerConfig. For example, if you set beatsPerMinute as
60
, the SDK plays one beat every second. If the file duration exceeds the beat duration, the SDK only plays the audio within the beat duration. - By default, the sound of the virtual metronome is published in the channel. If you do not want the sound to be heard by the remote users, you can set publishRhythmPlayerTrack in ChannelMediaOptions as
false
.
Parameters
- sound1
- The absolute path or URL address (including the filename extensions) of the file for the downbeat. For example,
C:\music\audio.mp4
. For the audio file formats supported by this method, see What formats of audio files does the Agora RTC SDK support. - sound2
- The absolute path or URL address (including the filename extensions) of the file for the upbeats. For example,
C:\music\audio.mp4
. For the audio file formats supported by this method, see What formats of audio files does the Agora RTC SDK support. - config
- The metronome configuration. See AgoraRhythmPlayerConfig.
Returns
- 0: Success.
- < 0: Failure.
- -22: Cannot find audio effect files. Please set the correct paths for sound1 and sound2.
stopAudioMixing
Stops playing and mixing the music file.
virtual int stopAudioMixing() = 0;
This method stops the audio mixing. Call this method when you are in a channel.
Returns
- 0: Success.
- < 0: Failure.
stopAudioRecording
Stops the audio recording on the client.
virtual int stopAudioRecording() = 0;
Returns
- 0: Success.
- < 0: Failure.
stopRhythmPlayer
Disables the virtual metronome.
virtual int stopRhythmPlayer() = 0;
After calling startRhythmPlayer, you can call this method to disable the virtual metronome.
Returns
- 0: Success.
- < 0: Failure.
onActiveSpeaker
Occurs when the most active remote speaker is detected.
virtual void onActiveSpeaker(uid_t userId) { (void)userId; }
After a successful call of enableAudioVolumeIndication, the SDK continuously detects which remote user has the loudest volume. During the current period, the remote user, who is detected as the loudest for the most times, is the most active user.
- If the most active remote speaker is always the same user, the SDK triggers the onActiveSpeaker callback only once.
- If the most active remote speaker changes to another user, the SDK triggers this callback again and reports the uid of the new active remote speaker.
Parameters
- userId
- The user ID of the most active remote speaker.
onAudioMixingFinished
Occurs when the playback of the local music file finishes.
virtual void onAudioMixingFinished() { }
- Deprecated:
- Please use onAudioMixingStateChanged instead.
After you call startAudioMixing [2/2] to play a local music file, this callback occurs when the playback finishes. If the call startAudioMixing [2/2] fails, the error code WARN_AUDIO_MIXING_OPEN_ERROR is returned.
onAudioMixingStateChanged
Occurs when the playback state of the music file changes.
virtual void onAudioMixingStateChanged(AUDIO_MIXING_STATE_TYPE state, AUDIO_MIXING_REASON_TYPE reason) { (void)state; (void)reason;
This callback occurs when the playback state of the music file changes, and reports the current state and error code.
Parameters
- state
- The playback state of the music file. See AUDIO_MIXING_STATE_TYPE.
- reason
- Error code. See AUDIO_MIXING_REASON_TYPE.
onAudioPublishStateChanged
Occurs when the audio publishing state changes.
virtual void onAudioPublishStateChanged(const char* channel, STREAM_PUBLISH_STATE oldState, STREAM_PUBLISH_STATE newState, int elapseSinceLastState) { (void)channel; (void)oldState; (void)newState; (void)elapseSinceLastState; }
Parameters
- channel
- The channel name.
- oldState
- The previous subscribing status. See STREAM_PUBLISH_STATE.
- newState
- The current subscribing status. See STREAM_PUBLISH_STATE.
- elapseSinceLastState
- The time elapsed (ms) from the previous state to the current state.
onAudioQuality
Reports the statistics of the audio stream from each remote user.
virtual void onAudioQuality(uid_t uid, int quality, unsigned short delay, unsigned short lost) { (void)uid; (void)quality; (void)delay; (void)lost; }
- Deprecated:
- Please use onRemoteAudioStats instead.
The SDK triggers this callback once every two seconds to report the audio quality of each remote user/host sending an audio stream. If a channel has multiple users/hosts sending audio streams, the SDK triggers this callback as many times.
Parameters
- uid
- The user ID of the remote user sending the audio stream.
- quality
- Audio quality of the user. See QUALITY_TYPE.
- delay
- The network delay (ms) from the sender to the receiver, including the delay caused by audio sampling pre-processing, network transmission, and network jitter buffering.
- lost
- The packet loss rate (%) of the audio packet sent from the remote user.
onAudioRoutingChanged
Occurs when the local audio route changes.
virtual void onAudioRoutingChanged(int routing) { (void)routing; }
This method is for Android, iOS and macOS only.
Parameters
- routing
-
The current audio routing. See AudioRoute.
onAudioVolumeIndication
Reports the volume information of users.
virtual void onAudioVolumeIndication(const AudioVolumeInfo* speakers, unsigned int speakerNumber, int totalVolume) { (void)speakers; (void)speakerNumber; (void)totalVolume; }
By default, this callback is disabled. You can enable it by calling enableAudioVolumeIndication. Once this callback is enabled and users send streams in the channel, the SDK triggers the onAudioVolumeIndication callback according to the time interval set in enableAudioVolumeIndication. The SDK triggers two independent onAudioVolumeIndication callbacks simultaneously, which separately report the volume information of the local user who sends a stream and the remote users (up to three) whose instantaneous volume is the highest.
Once this callback is enabled, if the local user calls the muteLocalAudioStream method for mute, the SDK continues to report the volume indication of the local user.
20 seconds after a remote user whose volume is one of the three highest in the channel stops publishing the audio stream, the callback excludes this user's information; 20 seconds after all remote users stop publishing audio streams, the SDK stops triggering the callback for remote users.
Parameters
- speakers
- The volume information of the users. See AudioVolumeInfo. An empty speakers array in the callback indicates that no remote user is in the channel or sending a stream at the moment.
- speakerNumber
-
The total number of users.
- In the callback for the local user, if the local user is sending streams, the value of speakerNumber is 1.
- In the callback for remote users, the value range of speakerNumber is [0,3]. If the number of remote users who send streams is greater than or equal to three, the value of speakerNumber is 3.
- totalVolume
-
The volume of the speaker. The value ranges between 0 (lowest volume) and 255 (highest volume).
- In the callback for the local user, totalVolume is the volume of the local user who sends a stream.
- In the callback for remote users, totalVolume is the sum of the volume of the remote users (up to three) whose instantaneous volume are the highest. If the user calls startAudioMixing [2/2], totalVolume is the volume after audio mixing.
onFirstLocalAudioFramePublished
Occurs when the first audio frame is published.
virtual void onFirstLocalAudioFramePublished(int elapsed) { (void)elapsed; }
- The local client enables the audio module and calls joinChannel [2/2] successfully.
- The local client calls
muteLocalAudioStream(
andtrue
)muteLocalAudioStream(
in sequence.false
) - The local client calls disableAudio and enableAudio in sequence.
- The local client calls pushAudioFrame to successfully push the audio frame to the SDK.
Parameters
- elapsed
- Time elapsed (ms) from the local user calling joinChannel [2/2] until the SDK triggers this callback.
onFirstRemoteAudioDecoded
Occurs when the SDK decodes the first remote audio frame for playback.
virtual void onFirstRemoteAudioDecoded(uid_t uid, int elapsed) { (void)uid; (void)elapsed; }
- Deprecated:
- Use onRemoteAudioStateChanged instead.
- The remote user joins the channel and sends the audio stream.
- The remote user stops sending the audio stream and re-sends it after 15 seconds, and the possible reasons include:
- The remote user leaves the channel.
- The remote user is offline.
- The remote user calls muteLocalAudioStream to stop sending the video stream.
- The remote user calls disableAudio to disable video.
Parameters
- uid
- The user ID of the remote user.
- elapsed
- The time elapsed (ms) from the local user calling the joinChannel [2/2] method until the SDK triggers this callback.
onFirstRemoteAudioFrame
Occurs when the first audio frame sent by a specified remote user is received.
virtual void onFirstRemoteAudioFrame(uid_t uid, int elapsed) AGORA_DEPRECATED_ATTRIBUTE { (void)uid; (void)elapsed; }
- Deprecated:
- Use onRemoteAudioStateChanged instead.
Parameters
- uid
- The ID of the remote user sending the audio frames.
- elapsed
- The time elapsed (ms) from the local user calling the joinChannel [2/2] method until the SDK triggers this callback.
onLocalAudioStateChanged
Occurs when the local audio stream state changes.
virtual void onLocalAudioStateChanged(LOCAL_AUDIO_STREAM_STATE state, LOCAL_AUDIO_STREAM_ERROR error) { (void)state; (void)error; }
When the state of the local audio stream changes (including the state of the audio capture and encoding), the SDK triggers this callback to report the current state. This callback indicates the state of the local audio stream, and allows you to troubleshoot issues when audio exceptions occur.
Parameters
- state
- The state of the local audio. See LOCAL_AUDIO_STREAM_STATE.
- error
- Local audio state error codes. See LOCAL_AUDIO_STREAM_ERROR.
onLocalAudioStats
Reports the statistics of the local audio stream.
virtual void onLocalAudioStats(const LocalAudioStats& stats) { (void)stats; }
The SDK triggers this callback once every two seconds.
Parameters
- stats
- Local audio statistics. See LocalAudioStats.
onRemoteAudioStateChanged
Occurs when the remote audio state changes.
virtual void onRemoteAudioStateChanged(uid_t uid, REMOTE_AUDIO_STATE state, REMOTE_AUDIO_STATE_REASON reason, int elapsed) { (void)uid; (void)state; (void)reason; (void)elapsed; }
When the audio state of a remote user (in a voice/video call channel) or host (in a live streaming channel) changes, the SDK triggers this callback to report the current state of the remote audio stream.
Parameters
- uid
- The ID of the remote user whose audio state changes.
- state
- The state of the remote audio. See REMOTE_AUDIO_STATE.
- reason
- The reason of the remote audio state change. See REMOTE_AUDIO_STATE_REASON.
- elapsed
- Time elapsed (ms) from the local user calling the joinChannel [2/2] method until the SDK triggers this callback.
onRemoteAudioStats
Reports the statistics of the audio stream sent by each remote users.
virtual void onRemoteAudioStats(const RemoteAudioStats& stats) { (void)stats; }
The SDK triggers this callback once every two seconds. If a channel includes multiple users, the SDK triggers this callback as many times.
Parameters
- stats
- Statistics of the received remote audio stream. See RemoteAudioStats.
onRemoteAudioTransportStats
Reports the transport-layer statistics of each remote audio stream.
virtual void onRemoteAudioTransportStats(uid_t uid, unsigned short delay, unsigned short lost, unsigned short rxKBitRate) { (void)uid; (void)delay; (void)lost; (void)rxKBitRate; }
- Deprecated:
- Please use onRemoteAudioStats instead.
This callback reports the transport-layer statistics, such as the packet loss rate and network time delay, once every two seconds after the local user receives an audio packet from a remote user. During a call, when the user receives the video packet sent by the remote user/host, the callback is triggered every 2 seconds.
Parameters
- uid
- The ID of the remote user sending the audio packets.
- delay
- The network delay (ms) from the sender to the receiver.
- lost
- The packet loss rate (%) of the audio packet sent from the remote user.
- rxKBitRate
- The bitrate of the received audio (Kbps).
onRhythmPlayerStateChanged
Occurs when the state of virtual metronome changes.
virtual void onRhythmPlayerStateChanged(RHYTHM_PLAYER_STATE_TYPE state, RHYTHM_PLAYER_ERROR_TYPE errorCode) { (void)state; (void)errorCode; }
When the state of the virtual metronome changes, the SDK triggers this callback to report the current state of the virtual metronome. This callback indicates the state of the local audio stream and enables you to troubleshoot issues when audio exceptions occur.
Parameters
- state
- For the current virtual metronome status, see RHYTHM_PLAYER_STATE_TYPE.
- errorCode
- For the error codes and error messages related to virtual metronome errors, see RHYTHM_PLAYER_ERROR_TYPE.
onUserMuteAudio
Occurs when a remote user (in the communication profile) or a host (in the live streaming profile) stops/resumes sending the audio stream.
virtual void onUserMuteAudio(uid_t uid, bool muted) { (void)uid; (void)muted; }
The SDK triggers this callback when the remote user stops or resumes sending the audio stream by calling the muteLocalAudioStream method.
Parameters
- uid
- The user ID.
- muted
- Whether the remote user's audio stream is muted/unmuted:
true
: User's audio stream is muted.false
: User's audio stream is unmuted.