本文以接收 PCM 格式的音频流和 YUV 格式的视频流为例,介绍从声网的 SD-RTN™ 接收媒体流的基本流程以及集成过程中的注意事项。
下图分别展示了接收远端音频和视频数据的数据处理和流转过程:
其中:
IVideoSinkBase
也可以用于渲染远端视频。首先,你需要在你的应用程序里调用 createAgoraService
和 initialize
进行全局初始化,创建并初始化 AgoraService
对象。
这个操作只需要进行一次,AgoraService
对象的生命期和应用程序的生命期保持一致,只要应用程序没有退出,AgoraService
可以一直存在。
createAndInitAgoraService
为 wrapper 函数,实现代码可参考 SDK 包 agora_sdk_demo/samples/common
路径下的 sample_common.cpp
文件。createAndInitAgoraService
时,enableAudioDevice
必须为 false
,否则会报错。 auto service = createAndInitAgoraService(false, true, true);
初始化后,根据以下步骤让本地 AgoraService
对象与声网服务器建立连接。
调用 createRtcConnection
创建 IRtcConnection
对象,用于与声网服务器建立连接。
(可选)你可以通过 VideoSubscriptionOptions
和 AudioSubscriptionOptions
定义音视频订阅选项。
创建 IRtcConnectionObserver
对象(即示例代码中的 SampleConnectionObserver
对象),用于监测连接事件。然后通过 registerObserver
方法将 IRtcConnectionObserver
对象和 IRtcConnection
对象关联起来。
调用 connect
与声网服务器建立连接。
// Configure audio subscription options
agora::rtc::AudioSubscriptionOptions audioSubOpt;
audioSubOpt.bytesPerSample = sizeof(int16_t) * options.audio.numOfChannels;
audioSubOpt.numberOfChannels = options.audio.numOfChannels;
audioSubOpt.sampleRateHz = options.audio.sampleRate;
// Configure connection options
agora::rtc::RtcConnectionConfiguration ccfg;
ccfg.clientRoleType = agora::rtc::CLIENT_ROLE_AUDIENCE;
ccfg.audioSubscriptionOptions = audioSubOpt;
ccfg.autoSubscribeAudio = false;
ccfg.autoSubscribeVideo = false;
ccfg.enableAudioRecordingOrPlayout =
false; // Subscribe audio but without playback
// Create声网connection
agora::agora_refptr<agora::rtc::IRtcConnection> connection =
service->createRtcConnection(ccfg);
if (!connection) {
AG_LOG(ERROR, "Failed to creating声网connection!");
return -1;
}
// Subcribe streams from all remote users or specific remote user
agora::rtc::ILocalUser::VideoSubscriptionOptions subscriptionOptions;
if (options.streamType == STREAM_TYPE_HIGH) {
subscriptionOptions.type = agora::rtc::REMOTE_VIDEO_STREAM_HIGH;
} else if(options.streamType==STREAM_TYPE_LOW){
subscriptionOptions.type = agora::rtc::REMOTE_VIDEO_STREAM_LOW;
} else{
AG_LOG(ERROR, "It is a error stream type");
return -1;
}
if (options.remoteUserId.empty()) {
AG_LOG(INFO, "Subscribe streams from all remote users");
connection->getLocalUser()->subscribeAllAudio();
connection->getLocalUser()->subscribeAllVideo(subscriptionOptions);
} else {
connection->getLocalUser()->subscribeAudio(options.remoteUserId.c_str());
connection->getLocalUser()->subscribeVideo(options.remoteUserId.c_str(),
subscriptionOptions);
}
// Register connection observer to monitor connection event
auto connObserver = std::make_shared<SampleConnectionObserver>();
connection->registerObserver(connObserver.get());
// Connect to声网channel
if (connection->connect(options.appId.c_str(), options.channelId.c_str(),
options.userId.c_str())) {
AG_LOG(ERROR, "Failed to connect to声网channel!");
return -1;
}
ILocalUserObserver
对象(即示例代码中的 SampleLocalUserObserver
对象)。setAudioFrameObserver
基于 registerAudioFrameObserver
实现,实现代码可参考 SDK 包 agora_sdk_demo/samples/common 路径下的 sample_local_user_observer.cpp
文件。setVideoFrameObserver
基于 addRenderer
实现,实现代码可参考 SDK 包 agora_sdk_demo/samples/common 路径下的 sample_local_user_observer.cpp
文件。// Create local user observer
auto localUserObserver =
std::make_shared<SampleLocalUserObserver>(connection->getLocalUser());
// Register audio frame observer to receive audio stream
auto pcmFrameObserver = std::make_shared<PcmFrameObserver>(options.audioFile);
if (connection->getLocalUser()->setPlaybackAudioFrameBeforeMixingParameters(
options.audio.numOfChannels, options.audio.sampleRate)) {
AG_LOG(ERROR, "Failed to set audio frame parameters!");
return -1;
}
localUserObserver->setAudioFrameObserver(pcmFrameObserver.get());
// Register video frame observer to receive video stream
agora::agora_refptr<agora::rtc::IVideoSinkBase> yuvFrameObserver =
new agora::RefCountedObject<YuvFrameObserver>(options.videoFile);
localUserObserver->setVideoFrameObserver(yuvFrameObserver);
注册音频帧观测器和视频帧观测器后,当 SDK 触发 onPlaybackAudioFrameBeforeMixing
或 onFrame
回调时,意味着收到了一个音频帧或视频帧。你需要在这两个回调函数中添加接收媒体流的逻辑,比如将音频帧写入音频文件。
// Callback to receive a PCM frame
bool PcmFrameObserver::onPlaybackAudioFrameBeforeMixing(
unsigned int uid, AudioFrame& audioFrame) {
// Create new file to save received PCM samples
if (!pcmFile_) {
std::string fileName = (++fileCount > 1)
? (outputFilePath_ + to_string(fileCount))
: outputFilePath_;
if (!(pcmFile_ = fopen(fileName.c_str(), "w"))) {
AG_LOG(ERROR, "Failed to create received audio file %s",
fileName.c_str());
return false;
}
AG_LOG(INFO, "Created file %s to save received PCM samples",
fileName.c_str());
}
// Write PCM samples
size_t writeBytes =
audioFrame.samplesPerChannel * audioFrame.channels * sizeof(int16_t);
if (fwrite(audioFrame.buffer, 1, writeBytes, pcmFile_) != writeBytes) {
AG_LOG(ERROR, "Error writing decoded audio data: %s", std::strerror(errno));
return false;
}
fileSize_ += writeBytes;
// Close the file if size limit is reached
if (fileSize_ >= DEFAULT_FILE_LIMIT) {
fclose(pcmFile_);
pcmFile_ = nullptr;
fileSize_ = 0;
}
return true;
}
// Callback to receive a YUV frame
int YuvFrameObserver::onFrame(
const agora::media::base::VideoFrame& videoFrame) {
// Create new file to save received YUV frames
if (!yuvFile_) {
std::string fileName = (++fileCount > 1)
? (outputFilePath_ + to_string(fileCount))
: outputFilePath_;
if (!(yuvFile_ = fopen(fileName.c_str(), "w+"))) {
AG_LOG(ERROR, "Failed to create received video file %s",
fileName.c_str());
return -1;
}
AG_LOG(INFO, "Created file %s to save received YUV frames",
fileName.c_str());
}
// Write Y planar
size_t writeBytes = videoFrame.yStride * videoFrame.height;
if (fwrite(videoFrame.yBuffer, 1, writeBytes, yuvFile_) != writeBytes) {
AG_LOG(ERROR, "Error writing decoded video data: %s", std::strerror(errno));
return -1;
}
fileSize_ += writeBytes;
// Write U planar
writeBytes = videoFrame.uStride * videoFrame.height / 2;
if (fwrite(videoFrame.uBuffer, 1, writeBytes, yuvFile_) != writeBytes) {
AG_LOG(ERROR, "Error writing decoded video data: %s", std::strerror(errno));
return -1;
}
fileSize_ += writeBytes;
// Write V planar
writeBytes = videoFrame.vStride * videoFrame.height / 2;
if (fwrite(videoFrame.vBuffer, 1, writeBytes, yuvFile_) != writeBytes) {
AG_LOG(ERROR, "Error writing decoded video data: %s", std::strerror(errno));
return -1;
}
fileSize_ += writeBytes;
// Close the file if size limit is reached
if (fileSize_ >= DEFAULT_FILE_LIMIT) {
fclose(yuvFile_);
yuvFile_ = nullptr;
fileSize_ = 0;
}
return 0;
};
媒体流传输完成后,你可以通过以下步骤退出频道并断开与声网服务器的连接:
disconnect
断开与声网服务器的连接。IRtcConnection
对象相关的资源。 // Unregister audio & video frame observers
localUserObserver->unsetAudioFrameObserver();
localUserObserver->unsetVideoFrameObserver();
// Disconnect from声网channel
if (connection->disconnect()) {
AG_LOG(ERROR, "Failed to disconnect from声网channel!");
return -1;
}
AG_LOG(INFO, "Disconnected from声网channel successfully");
// Destroy声网connection and related resources
localUserObserver.reset();
pcmFrameObserver.reset();
yuvFrameObserver = nullptr;
connection = nullptr;
当你的应用程序退出时,可以调用如下逻辑,注销整个 AgoraService
。
// Destroy声网 Service
service->release();
service = nullptr;
Linux SDK 支持使用 String 型用户 ID 加入频道。使用时,你需要注意以下限制:
createAndInitAgoraService
时,需要将 enableuseStringUid
设为 true
。connect
方法时,你必须给 userId
赋值。设置为 null
会收到报错。具体代码可参考 sample_stringuid_receive.cpp
示例项目。
你可以使用 encryptionMode
参数设置加密算法以及加密密钥。具体代码可参考 sample_receive_decrypted_h264.cpp
示例项目。
// Set the encryption mode
if (options.encryptionMode == true) {
agora::rtc::EncryptionConfig Config;
Config.encryptionMode = agora::rtc::SM4_128_ECB;
Config.encryptionKey = options.encryptionKey.c_str();
if (connection->enableEncryption(options.encryptionMode, Config) < 0) {
AG_LOG(ERROR, "Failed to enable encryption!");
}
AG_LOG(INFO, "Enable encryption successfully");
}
LIVE_BROADCASTING
。我们在 SDK 中提供了一系列接收媒体流的示例项目,供你参考:
示例项目 | 实现功能 |
---|---|
sample_receive_yuv_pcm.cpp |
接收 YUV 帧以及混音前的 PCM 帧。 |
sample_receive_h264_pcm.cpp |
接收 H.264 帧以及混音前的 PCM 帧。 |
sample_receive_mixed_audio.cpp |
接收混音后的 PCM 帧。 |
sample_receive_yuv_from_multi_user.cpp |
接收多路 YUV 流。 |
sample_receive_decrypted_h264.cpp |
接收加密后的 H.264 帧。 |
sample_stringuid_receive.cpp |
使用 String 型用户 ID 加入频道并接收流。 |