Before reading this article, you need to undersand two concepts: audio source and audio sink. Audio source is where audio frames come from, such as a microphone and a recorder. Audio sink is where audio frames go, such as a speaker and a earpiece.
By default, the Agora SDK uses default audio and video modules for capturing and rendering in real-time communications.
However, the default modules might not meet your development requirements, such as in the following scenarios:
This article tells you how to use the Agora Unity SDK to customize the audio source and sink.
Before customizing the audio source or sink, ensure that you implement the basic real-time communication functions in your project. For details, see Start a Call or Start Live Interactive Streaming.
Refer to the following steps to customize the audio source in your project:
SetExternalAudioSource
method to enable the external audio source before joining a channel.PushAudioFrame
method.Sample code
// Notifies the SDK that you want to use the external audio sink.
public int SetExternalAudioSource(bool enabled, int sampleRate, int channels)
{
mRtcEngine.SetExternalAudioSource(enabled, sampleRate, channels);
}
// Pushes the remote audio data for playback.
public int PushAudioFrame(AudioFrame audioFrame)
{
mRtcEngine.PushAudioFrame(audioFrame);
}
public struct AudioFrame
{
// The type of the audio frame. See #AUDIO_FRAME_TYPE.
public AUDIO_FRAME_TYPE type;
// The number of samples per channel in the audio frame.
public int samples;
// The number of bytes per audio sample, which is usually 16-bit (2-byte).
public int bytesPerSample;
// The number of audio channels.
// - 1: Mono
// - 2: Stereo (the data is interleaved)
public int channels;
// The sample rate.
public int samplesPerSec;
// The data buffer of the audio frame. When the audio frame uses a stereo channel, the data buffer is interleaved.
// The size of the data buffer is as follows: buffer = samples × channels × bytesPerSample.
public byte[] buffer;
// The timestamp of the external audio frame. You can use this parameter for the following purposes:
// - Restore the order of the captured audio frame.
// - Synchronize audio and video frames in video-related scenarios, including where external video sources are used.
public long renderTimeMs;
// Reserved for future use.
public int avsync_type;
};
API reference
Refer to the following steps to customize the audio sink in your project:
SetExternalAudioSink
method to enable the external audio sink before joining a channel.PullAudioFrame
method to get the remote audio data.Sample code
// Notifies the SDK that you want to use the external audio sink.
public int SetExternalAudioSink(bool enabled, int sampleRate, int channels)
{
mRtcEngine.SetExternalAudioSink(enabled, sampleRate, channels);
}
// Pulls the remote audio data for playback.
public int PullAudioFrame(IntPtr audioBuffer, int type, int samples, int bytesPerSample, int channels, int samplesPerSec, long renderTimeMs, int avsync_type)
{
mRtcEngine.PullAudioFrame(audioBuffer, type, samples, bytesPerSample, channels, samplesPerSec, renderTimeMs, avsync_type);
}
API reference
Ensure the accuracy and efficiency of the audio data processing in the callback methods to avoid any possible crash.
Set the audio data to RAW_AUDIO_FRAME_OP_MODE_READ_WRITE
if you want to read, write, and manipulate the data.
Customizing the audio source and sink requires you to manage audio data recording and playback on your own.