Video Processing
Introduces methods and callbacks related to video processing.
addVideoWatermark [1/2]
Adds a watermark image to the local video.
virtual int addVideoWatermark(const RtcImage& watermark) = 0;
- Deprecated:
- This method is deprecated. Use addVideoWatermark [2/2] instead.
This method adds a PNG watermark image to the local video stream in a live streaming session. Once the watermark image is added, all the users in the channel (CDN audience included) and the video capturing device can see and capture it. If you only want to add a watermark to the CDN live streaming, see descriptions in setLiveTranscoding.
- The URL descriptions are different for the local video and CDN live streaming: In a local video stream, URL refers to the absolute path of the added watermark image file in the local video stream. In a CDN live stream, URL refers to the URL address of the added watermark image in the CDN live streaming.
- The source file of the watermark image must be in the PNG file format. If the width and height of the PNG file differ from your settings in this method, the PNG file will be cropped to conform to your settings.
- The Agora SDK supports adding only one watermark image onto a local video or CDN live stream. The newly added watermark image replaces the previous one.
Parameters
- watermark
- The watermark image to be added to the local live streaming: RtcImage.
Returns
- 0: Success.
- < 0: Failure.
addVideoWatermark [2/2]
Adds a watermark image to the local video.
virtual int addVideoWatermark(const char* watermarkUrl, const WatermarkOptions& options) = 0;
This method adds a PNG watermark image to the local video in the live streaming. Once the watermark image is added, all the audience in the channel (CDN audience included), and the capturing device can see and capture it. Agora supports adding only one watermark image onto the local video, and the newly watermark image replaces the previous one.
- If the orientation mode of the encoding video (ORIENTATION_MODE) is fixed landscape mode or the adaptive landscape mode, the watermark uses the landscape orientation.
- If the orientation mode of the encoding video (ORIENTATION_MODE) is fixed portrait mode or the adaptive portrait mode, the watermark uses the portrait orientation.
- When setting the watermark position, the region must be less than the dimensions set in the setVideoEncoderConfiguration method; otherwise, the watermark image will be cropped.
- Ensure that call this method after enableVideo.
- If you only want to add a watermark to the media push, you can call this method or the setLiveTranscoding method.
- This method supports adding a watermark image in the PNG file format only. Supported pixel formats of the PNG image are RGBA, RGB, Palette, Gray, and Alpha_gray.
- If the dimensions of the PNG image differ from your settings in this method, the image will be cropped or zoomed to conform to your settings.
- If you have enabled the local video preview by calling the startPreview [1/2] method, you can use the
visibleInPreview
member to set whether or not the watermark is visible in the preview. - If you have enabled the mirror mode for the local video, the watermark on the local video is also mirrored. To avoid mirroring the watermark, Agora recommends that you do not use the mirror and watermark functions for the local video at the same time. You can implement the watermark function in your application layer.
Parameters
- watermarkUrl
- The local file path of the watermark image to be added. This method supports adding a watermark image from the local absolute or relative file path.
- options
- The options of the watermark image to be added. For details, see WatermarkOptions.
Returns
- 0: Success.
- < 0: Failure.
clearVideoWatermarks
Removes the watermark image from the video stream.
virtual int clearVideoWatermarks() = 0;
Returns
- 0: Success.
- < 0: Failure.
createCustomVideoTrack
Creates a customized video track.
virtual video_track_id_t createCustomVideoTrack() = 0;
- Call this method to create a video track and get the video track ID.
- In each channel's ChannelMediaOptions, set the customVideoTrackId parameter to the ID of the video track you want to publish, and set publishCustomVideoTrack to
true
. - If you call pushVideoFrame, and specify customVideoTrackId as the videoTrackId set in step 2, you can publish the corresponding custom video source in multiple channels.
Returns
- If the method call is successful, the video track ID is returned as the unique identifier of the video track.
- If the method call fails, a negative value is returned.
destroyCustomVideoTrack
Destroys the specified video track.
virtual int destroyCustomVideoTrack(video_track_id_t video_track_id) = 0;
Parameters
- video_track_id
- The video track ID returned by calling the createCustomVideoTrack method.
Returns
- 0: Success.
- < 0: Failure.
disableVideo
Disables the video module.
virtual int disableVideo() = 0;
This method disables video. You can call this method either before or after joining a channel. If you call it before joining a channel, an audio call starts when you join the channel. If you call it after joining a channel, a video call switches to an audio call. Call enableVideo to enable video.
A successful call of this method triggers the onUserEnableVideo (false
) callback on the remote client.
- This method affects the internal engine and can be called after leaving the channel.
- This method resets the internal engine and takes some time to take effect. Agora recommends using the following API methods to control the video engine modules separately:
- enableLocalVideo: Whether to enable the camera to create the local video stream.
- muteLocalVideoStream: Whether to publish the local video stream.
- muteRemoteVideoStream: Whether to subscribe to and play the remote video stream.
- muteAllRemoteVideoStreams: Whether to subscribe to and play all remote video streams.
Returns
- 0: Success.
- < 0: Failure.
enableContentInspect
Enables or disables video screenshot and upload.
virtual int enableContentInspect(bool enabled, const media::ContentInspectConfig &config) = 0;
When video screenshot and upload function is enabled, the SDK takes screenshots and upload videos sent by local users based on the type and frequency of the module you set in ContentInspectConfig. After video screenshot and upload, the Agora server sends the callback notification to your app server in HTTPS requests and sends all screenshots to the third-party cloud storage service.
- Before calling this method, ensure that you contact technical support to enbale Agora video screenshot and upload service.
- This method relies on the video content moderation library
libagora_ci_extension.dll
. If the dynamic library is deleted, the function cannot be enabled normally.
Parameters
- enabled
- Whether to enable video screenshot and upload:
true
: Enables video screenshot and upload.false
: Disables video screenshot and upload.
- config
- Configuration of video screenshot and upload. See ContentInspectConfig.
Returns
- 0: Success.
- < 0: Failure.
enableFaceDetection
Enables/Disables face detection for the local user.
virtual int enableFaceDetection(bool enabled) = 0;
You can call this method either before or after joining a channel.
- The width and height of the local video.
- The position of the human face in the local view.
- The distance between the human face and the screen.
This method needs to be called after the camera is started (for example, by calling startPreview [1/2] or joinChannel [2/2]).
Parameters
- enabled
- Whether to enable face detection for the local user:
true
: Enable face detection.false
: (Default) Disable face detection.
Returns
- 0: Success.
- < 0: Failure.
enableLocalVideo
Enables/Disables the local video capture.
virtual int enableLocalVideo(bool enabled) = 0;
This method disables or re-enables the local video capturer, and does not affect receiving the remote video stream.
After calling enableVideo, the local video capturer is enabled by default. You can call enableLocalVideo(false
) to disable the local video capturer. If you want to re-enable the local video, call enableLocalVideo(true
).
After the local video capturer is successfully disabled or re-enabled, the SDK triggers the onRemoteVideoStateChanged callback on the remote client.
- You can call this method either before or after joining a channel.
- This method enables the internal engine and is valid after leaving the channel.
Parameters
- enabled
-
Whether to enable the local video capture.
true
: (Default) Enable the local video capture.false
: Disables the local video capture. Once the local video is disabled, the remote users can no longer receive the video stream of this user, while this user can still receive the video streams of the other remote users. When set tofalse
, this method does not require a local camera.
Returns
- 0: Success.
- < 0: Failure.
enableMultiCamera
Enables or disables multi-camera capture.
#if defined(__APPLE__) && TARGET_OS_IOS virtual int enableMultiCamera(bool enabled, const CameraCapturerConfiguration& config) = 0; #endif
- Since
- v4.1.0
- Call this method to enable multi-channel camera capture.
- Call startPreview [1/2] to start the local video preview.
- Call startSecondaryCameraCapture to start video capture with the second camera.
- Call joinChannelEx, and set publishSecondaryCameraTrack to
true
to publish the video stream captured by the second camera in the channel.
- Call stopSecondaryCameraCapture.
- Call this method with enabled set to
false
.
- If it is enabled before startPreview [1/2], the local video preview shows the image captured by the two cameras at the same time.
- If it is enabled after startPreview [1/2], the SDK stops the current camera capture first, and then enables the primary camera and the second camera. The local video preview appears black for a short time, and then automatically returns to normal.
This method applies to iOS only.
When using this function, ensure that the system version is 13.0 or later.
- iPhone XR
- iPhone XS
- iPhone XS Max
- iPad Pro 3rd generation and later
Parameters
- enabled
- Whether to enable multi-camera video capture mode:
true
: Enable multi-camera capture mode; the SDK uses multiple cameras to capture video.false
: Disable multi-camera capture mode; the SDK uses a single camera to capture video.
- config
- Capture configuration for the second camera. See CameraCapturerConfiguration.
Returns
- 0: Success.
- < 0: Failure.
enableRemoteSuperResolution
Enables/Disables the super resolution algorithm for a remote user's video stream.
virtual int enableRemoteSuperResolution(uid_t userId, bool enable) = 0;
This function can effectively improve the resolution of the remote video picture seen by the local user, that is, the width and height (pixels) of the video received by the specified remote user are enlarged to 2 times original size.
- If the parameter superResolutionType >0: Super resolution is enabled.
- If the parameter superResolutionType =0: Super resolution is not enabled.
The super resolution feature requires extra system resources. To balance the visual experience and system consumption, this feature can only be enabled for a single remote user. If the local user uses super resolution on Android, the original resolution of the remote user's video cannot exceed 640 × 360 pixels; if the local user uses super resolution on iOS, the original resolution of the remote user's video cannot exceed 640 × 480 pixels.
- This method applies to Android and iOS only.
- This method relies on the super resolution dynamic library
libagora_super_resolution_extension.so (Android); AgoraSuperResolutionExtension.xcframework (iOS)
. If the dynamic library is deleted, the function cannot be enabled normally. - Because this method has certain system performance requirements, Agora recommends that you use the following devices or better:
- Android:
- VIVO: V1821A, NEX S, 1914A, 1916A, 1962A, 1824BA, X60, X60 Pro
- OPPO: PCCM00, Find X3
- OnePlus: A6000
- Xiaomi: Mi 8, Mi 9, Mi 10, Mi 11, MIX3, Redmi K20 Pro
- SAMSUNG: SM-G9600, SM-G9650, SM-N9600, SM-G9708, SM-G960U, SM-G9750, S20, S21
- HUAWEI: SEA-AL00, ELE-AL00, VOG-AL00, YAL-AL10, HMA-AL00, EVR-AN00, nova 4, nova 5 Pro, nova 6 5G, nova 7 5G, Mate 30, Mate 30 Pro, Mate 40, Mate 40 Pro, P40, P40 Pro, Huawei M6, MatePad 10.8
- iOS:
- iPhone XR
- iPhone XS
- iPhone XS Max
- iPhone 11
- iPhone 11 Pro
- iPhone 11 Pro Max
- iPhone 12
- iPhone 12 mini
- iPhone 12 Pro
- iPhone 12 Pro Max
- iPhone 12 SE (2nd generation)
- iPad Pro 11-inch (3rd generation)
- iPad Pro 12.9-inch (3rd generation)
- iPad Air 3 (3rd generation)
- iPad Air 3 (4th generation)
- Android:
Parameters
- userId
- The user ID of the remote user.
- enable
- Whether to enable super resolution for the remote user’s video:
true
:Enable super resolution.false
: Disable super resolution.
Returns
- 0: Success.
- < 0: Failure.
enableVideo
Enables the video module.
virtual int enableVideo() = 0;
Call this method either before joining a channel or during a call. If this method is called before joining a channel, the call starts in the video mode. Call disableVideo to disable the video mode.
A successful call of this method triggers the onRemoteVideoStateChanged callback on the remote client.
- This method enables the internal engine and is valid after leaving the channel.
- This method resets the internal engine and takes some time to take effect. Agora recommends using the following API methods to control the video engine modules separately:
- enableLocalVideo: Whether to enable the camera to create the local video stream.
- muteLocalVideoStream: Whether to publish the local video stream.
- muteRemoteVideoStream: Whether to subscribe to and play the remote video stream.
- muteAllRemoteVideoStreams: Whether to subscribe to and play all remote video streams.
Returns
- 0: Success.
- < 0: Failure.
enableVideoImageSource
Sets whether to replace the current video feeds with images when publishing video streams.
virtual int enableVideoImageSource(bool enable, const ImageTrackOptions& options) = 0;
When publishing video streams, you can call this method to replace the current video feeds with custom images.
Once you enable this function, you can select images to replace the video feeds through the ImageTrackOptions parameter. If you disable this function, the remote users see the video feeds that you publish.
Parameters
- enable
- Whether to replace the current video feeds with custom images:
true
: Replace the current video feeds with custom images.false
: (Default) Do not replace the current video feeds with custom images.
- options
- Image configurations. See ImageTrackOptions.
Returns
- 0: Success.
- < 0: Failure.
enableVirtualBackground
Enables/Disables the virtual background.
virtual int enableVirtualBackground(bool enabled, VirtualBackgroundSource backgroundSource, SegmentationProperty segproperty, agora::media::MEDIA_SOURCE_TYPE type = agora::media::PRIMARY_CAMERA_SOURCE) = 0;
The virtual background function allows you to replace the original background image of the local user or to blur the background. After successfully enabling the virtual background function, all users in the channel can see the customized background.
Call this method before calling enableVideo or startPreview [1/2].
- This function requires a high-performance device. Agora recommends that you use this function on devices with the following chips:
- Snapdragon 700 series 750G and later
- Snapdragon 800 series 835 and later
- Dimensity 700 series 720 and later
- Kirin 800 series 810 and later
- Kirin 900 series 980 and later
- Devices with an i5 CPU and better
- Devices with an A9 chip and better, as follows:
- iPhone 6S and later
- iPad Air 3rd generation and later
- iPad 5th generation and later
- iPad Pro 1st generation and later
- iPad mini 5th generation and later
- Agora recommends that you use this function in scenarios that meet the following conditions:
- A high-definition camera device is used, and the environment is uniformly lit.
- There are few objects in the captured video. Portraits are half-length and unobstructed. Ensure that the background is a solid color that is different from the color of the user's clothing.
- This method relies on the video enhancement dynamic library
libagora_segmentation_extension.dll
. If the dynamic library is deleted, the function cannot be enabled normally.
Parameters
- enabled
- Whether to enable virtual background:
true
: Enable virtual background.false
: Disable virtual background.
- backgroundSource
- The custom background image. See VirtualBackgroundSource. To adapt the resolution of the custom background image to that of the video captured by the SDK, the SDK scales and crops the custom background image while ensuring that the content of the custom background image is not distorted.
- segproperty
- Processing properties for background images. See SegmentationProperty.
- type
- The type of the video source. See MEDIA_SOURCE_TYPE.Attention: In this method, this parameter supports only the following two settings:
- The default value is PRIMARY_CAMERA_SOURCE.
- If you want to use the second camera to capture video, set this parameter to SECONDARY_CAMERA_SOURCE.
Returns
- 0: Success.
- < 0: Failure.
- -1: The custom background image does not exist. Check the value of source in VirtualBackgroundSource.
- -2: The color format of the custom background image is invalid. Check the value of color in VirtualBackgroundSource.
- -3: The device does not support virtual background.
pushVideoFrame
Pushes the external raw video frame to the SDK.
virtual int pushVideoFrame(base::ExternalVideoFrame* frame, unsigned int videoTrackId = 0) = 0;
To push the unencoded external raw video frame to the SDK, call createCustomVideoTrack to get the video track ID, set customVideoTrackId as the video track ID you want to publish in the ChannelMediaOptions of each channel, and set publishCustomVideoTrack as true
.
Parameters
- frame
-
The external raw video frame to be pushed. See ExternalVideoFrame.
- videoTrackId
- The video track ID returned by calling the createCustomVideoTrack method. The default value is 0.
Returns
- 0: Success.
- < 0: Failure.
setBeautyEffectOptions
Sets the image enhancement options.
virtual int setBeautyEffectOptions(bool enabled, BeautyOptions options) = 0;
Enables or disables image enhancement, and sets the options.
- Call this method before calling enableVideo or startPreview [1/2].
- This method relies on the video enhancement dynamic library
libagora_clear_vision_extension.dll
. If the dynamic library is deleted, the function cannot be enabled normally.
Parameters
- enabled
- Whether to enable the image enhancement function:
true
: Enable the image enhancement function.false
: (Default) Disable the image enhancement function.
- options
- The image enhancement options. See BeautyOptions.
Returns
- 0: Success.
- < 0: Failure.
setColorEnhanceOptions
Sets color enhancement.
virtual int setColorEnhanceOptions(bool enabled, const ColorEnhanceOptions& options) = 0;
The video images captured by the camera can have color distortion. The color enhancement feature intelligently adjusts video characteristics such as saturation and contrast to enhance the video color richness and color reproduction, making the video more vivid.
You can call this method to enable the color enhancement feature and set the options of the color enhancement effect.
- Call this method after calling enableVideo.
- The color enhancement feature has certain performance requirements on devices. With color enhancement turned on, Agora recommends that you change the color enhancement level to one that consumes less performance or turn off color enhancement if your device is experiencing severe heat problems.
- Both this method and setExtensionProperty can enable color enhancement:
- When you use the SDK to capture video, Agora recommends this method (this method only works for video captured by the SDK).
- When you use an external video source to implement custom video capture, or send an external video source to the SDK, Agora recommends using setExtensionProperty.
- This method relies on the video enhancement dynamic library
libagora_clear_vision_extension.dll
. If the dynamic library is deleted, the function cannot be enabled normally.
Parameters
- enabled
- Whether to enable color enhancement:
true
Enable color enhancement.false
: (Default) Disable color enhancement.
- options
- The color enhancement options. See ColorEnhanceOptions.
- type
- The type of the video source. See MEDIA_SOURCE_TYPE.
Returns
- 0: Success.
- < 0: Failure.
setLocalRenderMode [1/2]
Sets the local video display mode.
virtual int setLocalRenderMode(media::base::RENDER_MODE_TYPE renderMode) = 0;
- Deprecated:
- This method is deprecated. Use setLocalRenderMode [2/2] instead.
Call this method to set the local video display mode. This method can be called multiple times during a call to change the display mode.
Parameters
- renderMode
-
The local video display mode. See RENDER_MODE_TYPE.
Returns
- 0: Success.
- < 0: Failure.
setLocalRenderMode [2/2]
Updates the display mode of the local video view.
virtual int setLocalRenderMode(media::base::RENDER_MODE_TYPE renderMode, VIDEO_MIRROR_MODE_TYPE mirrorMode) = 0;
After initializing the local video view, you can call this method to update its rendering and mirror modes. It affects only the video view that the local user sees, not the published local video stream.
- Ensure that you have called the setupLocalVideo method to initialize the local video view before calling this method.
- During a call, you can call this method as many times as necessary to update the display mode of the local video view.
Parameters
- renderMode
-
The local video display mode. See RENDER_MODE_TYPE.
- mirrorMode
-
The rendering mode of the local video view. See VIDEO_MIRROR_MODE_TYPE.
Attention: If you use a front camera, the SDK enables the mirror mode by default; if you use a rear camera, the SDK disables the mirror mode by default.
Returns
- 0: Success.
- < 0: Failure.
setLocalVideoMirrorMode
Sets the local video mirror mode.
virtual int setLocalVideoMirrorMode(VIDEO_MIRROR_MODE_TYPE mirrorMode) = 0;
- Deprecated:
- This method is deprecated.
Parameters
- mirrorMode
-
The local video mirror mode. See VIDEO_MIRROR_MODE_TYPE.
Returns
- 0: Success.
- < 0: Failure.
setLowlightEnhanceOptions
Sets low-light enhancement.
virtual int setLowlightEnhanceOptions(bool enabled, const LowlightEnhanceOptions& options) = 0;
The low-light enhancement feature can adaptively adjust the brightness value of the video captured in situations with low or uneven lighting, such as backlit, cloudy, or dark scenes. It restores or highlights the image details and improves the overall visual effect of the video.
You can call this method to enable the color enhancement feature and set the options of the color enhancement effect.
- Call this method after calling enableVideo.
- Dark light enhancement has certain requirements for equipment performance. The low-light enhancement feature has certain performance requirements on devices. If your device overheats after you enable low-light enhancement, Agora recommends modifying the low-light enhancement options to a less performance-consuming level or disabling low-light enhancement entirely.
- Both this method and setExtensionProperty can turn on low-light enhancement:
- When you use the SDK to capture video, Agora recommends this method (this method only works for video captured by the SDK).
- When you use an external video source to implement custom video capture, or send an external video source to the SDK, Agora recommends using setExtensionProperty.
- This method relies on the video enhancement dynamic library
libagora_clear_vision_extension.dll
. If the dynamic library is deleted, the function cannot be enabled normally.
Parameters
- enabled
- Whether to enable low-light enhancement function:
true
: Enable low-light enhancement function.false
: (Default) Disable low-light enhancement function.
- options
- The low-light enhancement options. See LowlightEnhanceOptions.
- type
- The type of the video source. See MEDIA_SOURCE_TYPE.
Returns
- 0: Success.
- < 0: Failure.
setRemoteRenderMode
Updates the display mode of the video view of a remote user.
virtual int setRemoteRenderMode(uid_t userId, RENDER_MODE_TYPE renderMode, VIDEO_MIRROR_MODE_TYPE mirrorMode) = 0;
After initializing the video view of a remote user, you can call this method to update its rendering and mirror modes. This method affects only the video view that the local user sees.
- Please call this method after initializing the remote view by calling the setupRemoteVideo method.
- During a call, you can call this method as many times as necessary to update the display mode of the video view of a remote user.
Parameters
- userId
- The user ID of the remote user.
- renderMode
-
The rendering mode of the remote user view. For details, see RENDER_MODE_TYPE.
- mirrorMode
-
The mirror mode of the remote user view. For details, see VIDEO_MIRROR_MODE_TYPE.
Returns
- 0: Success.
- < 0: Failure.
setupLocalVideo
Initializes the local video view.
virtual int setupLocalVideo(const VideoCanvas& canvas) = 0;
This method initializes the video view of a local stream on the local device. It affects only the video view that the local user sees, not the published local video stream. Call this method to bind the local video stream to a video view and to set the rendering and mirror modes of the video view.
After initialization, call this method to set the local video and then join the channel. The local video still binds to the view after you leave the channel. To unbind the local video from the view, set the view parameter as NULL.
- You can call this method either before or after joining a channel.
- To update the rendering or mirror mode of the local video view during a call, use the setLocalRenderMode [2/2] method.
Parameters
- canvas
- The local video view and settings. See VideoCanvas.
Returns
- 0: Success.
- < 0: Failure.
setupRemoteVideo
Initializes the video view of a remote user.
virtual int setupRemoteVideo(const VideoCanvas& canvas) = 0;
This method initializes the video view of a remote stream on the local device. It affects only the video view that the local user sees. Call this method to bind the remote video stream to a video view and to set the rendering and mirror modes of the video view.
You need to specify the ID of the remote user in this method. If the remote user ID is unknown to the application, set it after the app receives the onUserJoined callback.
To unbind the remote user from the view, set the view parameter to NULL.
Once the remote user leaves the channel, the SDK unbinds the remote user.
- To update the rendering or mirror mode of the remote video view during a call, use the setRemoteRenderMode method.
- If you use the Agora recording function, the recording client joins the channel as a placeholder client, triggering the onUserJoined callback. Do not bind the placeholder client to the app view because the placeholder client does not send any video streams. If your app does not recognize the placeholder client, bind the remote user to the view when the SDK triggers the onFirstRemoteVideoDecoded callback.
Parameters
- canvas
-
The remote video view and settings. See VideoCanvas.
Returns
- 0: Success.
- < 0: Failure.
setVideoDenoiserOptions
Sets video noise reduction.
virtual int setVideoDenoiserOptions(bool enabled, const VideoDenoiserOptions& options) = 0;
Underlit environments and low-end video capture devices can cause video images to contain significant noise, which affects video quality. In real-time interactive scenarios, video noise also consumes bitstream resources and reduces encoding efficiency during encoding.
You can call this method to enable the video noise reduction feature and set the options of the video noise reduction effect.
- Call this method after calling enableVideo.
- Video noise reduction has certain requirements for equipment performance. If your device overheats after you enable video noise reduction, Agora recommends modifying the video noise reduction options to a less performance-consuming level or disabling video noise reduction entirely.
- Both this method and setExtensionProperty can turn on video noise reduction function:
- When you use the SDK to capture video, Agora recommends this method (this method only works for video captured by the SDK).
- When you use an external video source to implement custom video capture, or send an external video source to the SDK, Agora recommends using setExtensionProperty.
- This method relies on the video enhancement dynamic library
libagora_clear_vision_extension.dll
. If the dynamic library is deleted, the function cannot be enabled normally.
Parameters
- enabled
- Whether to enable video noise reduction:
true
: Enable video noise reduction.false
: (Default) Disable video noise reduction.
- options
- The video noise reduction options. See VideoDenoiserOptions.
- type
- The type of the video source. See MEDIA_SOURCE_TYPE.
Returns
- 0: Success.
- < 0: Failure.
setVideoEncoderConfiguration
Sets the video encoder configuration.
virtual int setVideoEncoderConfiguration(const VideoEncoderConfiguration& config) = 0;
Sets the encoder configuration for the local video.
Parameters
- config
- Video profile. See VideoEncoderConfiguration.
Returns
- 0: Success.
- < 0: Failure.
startLocalVideoTranscoder
Starts the local video mixing.
virtual int startLocalVideoTranscoder(const LocalTranscoderConfiguration& config) = 0;
- In a live streaming scenario with cohosts or when using the Media Push function, you can locally mix the videos of multiple hosts into one.
- In scenarios where you capture multiple local video streams (for example, video captured by cameras, screen sharing streams, video files, or pictures), you can merge them into one video stream and then publish the mixed video stream after joining the channel.
Parameters
- config
-
Configuration of the local video mixing. See LocalTranscoderConfiguration.
Returns
- 0: Success.
- < 0: Failure.
startPreview [1/2]
Enables the local video preview.
virtual int startPreview() = 0;
- Call setupLocalVideo to set the local preview window.
- Call enableVideo to enable the video.
- The local preview enables the mirror mode by default.
- After the local video preview is enabled, if you call leaveChannel [1/2] to exit the channel, the local preview remains until you call stopPreview [1/2] to disable it.
Returns
- 0: Success.
- < 0: Failure.
startPreview [2/2]
Enables the local video preview and specifies the video source for the preview.
virtual int startPreview(VIDEO_SOURCE_TYPE sourceType) = 0;
- Call setupLocalVideo to set the local preview window.
- Call enableVideo to enable the video.
- The local preview enables the mirror mode by default.
- After the local video preview is enabled, if you call leaveChannel [1/2] to exit the channel, the local preview remains until you call stopPreview [2/2] to disable it.
- The video source type set in this method needs to be consistent with the video source type of VideoCanvas you set in setupLocalVideo.
Parameters
- sourceType
- The type of the video frame, see VIDEO_SOURCE_TYPE.
Returns
- 0: Success.
- < 0: Failure.
startPrimaryCameraCapture
Starts video capture with a primary camera.
virtual int startPrimaryCameraCapture(const CameraCapturerConfiguration& config) = 0;
Parameters
- config
-
The configuration of the video capture with a primary camera. See CameraCapturerConfiguration.
Returns
- 0: Success.
- < 0: Failure.
startSecondaryCameraCapture
Starts video capture with a secondary camera.
virtual int startSecondaryCameraCapture(const CameraCapturerConfiguration& config) = 0;
Parameters
- config
-
The configuration of the video capture with a primary camera. See CameraCapturerConfiguration.
Returns
- 0: Success.
- < 0: Failure.
stopLocalVideoTranscoder
Stops the local video mixing.
virtual int stopLocalVideoTranscoder() = 0;
After calling startLocalVideoTranscoder, call this method if you want to stop the local video mixing.
Returns
- 0: Success.
- < 0: Failure.
stopPreview [1/2]
Stops the local video preview.
virtual int stopPreview() = 0;
After calling startPreview [1/2] to start the preview, if you want to close the local video preview, please call this method.
Returns
- 0: Success.
- < 0: Failure.
stopPreview [2/2]
Stops the local video preview.
virtual int stopPreview(VIDEO_SOURCE_TYPE sourceType) = 0;
After calling startPreview [2/2] to start the preview, if you want to close the local video preview, please call this method.
Parameters
- sourceType
- The type of the video frame, see VIDEO_SOURCE_TYPE.
Returns
- 0: Success.
- < 0: Failure.
stopPrimaryCameraCapture
Stops capturing video through a primary camera.
virtual int stopPrimaryCameraCapture() = 0;
You can call this method to stop capturing video through the primary camera after calling the startPrimaryCameraCapture.
Returns
- 0: Success.
- < 0: Failure.
stopSecondaryCameraCapture
Stops capturing video through the second camera.
virtual int stopSecondaryCameraCapture() = 0;
startSecondaryCameraCaptureYou can call this method to stop capturing video through the second camera after calling the .
On the iOS platform, if you want to disable multi-camera capture, you need to call enableMultiCameraafter calling this method and set enabled to false
.
Returns
- 0: Success.
- < 0: Failure.
takeSnapshot
Takes a snapshot of a video stream.
virtual int takeSnapshot(uid_t uid, const char* filePath) = 0;
This method takes a snapshot of a video stream from the specified user, generates a JPG image, and saves it to the specified path.
The method is asynchronous, and the SDK has not taken the snapshot when the method call returns. After a successful method call, the SDK triggers the onSnapshotTaken callback to report whether the snapshot is successfully taken, as well as the details for that snapshot.
- Call this method after joining a channel.
- This method takes a snapshot of the published video stream specified in ChannelMediaOptions.
- If the user's video has been preprocessed, for example, watermarked or beautified, the resulting snapshot includes the pre-processing effect.
Parameters
- uid
- The user ID. Set uid as 0 if you want to take a snapshot of the local user's video.
- filePath
-
The local path (including filename extensions) of the snapshot. For example:
- Windows:
C:\Users\<user_name>\AppData\Local\Agora\<process_name>\example.jpg
- iOS:
/App Sandbox/Library/Caches/example.jpg
- macOS:
~/Library/Logs/example.jpg
- Android:
/storage/emulated/0/Android/data/<package name>/files/example.jpg
- Windows:
Returns
- 0: Success.
- < 0: Failure.
onFacePositionChanged
Reports the face detection result of the local user.
virtual void onFacePositionChanged(int imageWidth, int imageHeight, const Rectangle* vecRectangle, const int* vecDistance, int numFaces) { (void) imageWidth; (void) imageHeight; (void) vecRectangle; (void) vecDistance; (void) numFaces; }
enableFaceDetection(true)
, you can get the following information on the local user in real-time:- The width and height of the local video.
- The position of the human face in the local view.
- The distance between the human face and the screen.
This value is based on the fitting calculation of the local video size and the position of the human face.
- This callback is for Android and iOS only.
- When it is detected that the face in front of the camera disappears, the callback will be triggered immediately. When no human face is detected, the frequency of this callback to be rtriggered wil be decreased to reduce power consumption on the local device.
- The SDK stops triggering this callback when a human face is in close proximity to the screen.
- On Android, the value of distance reported in this callback may be slightly different from the actual distance. Therefore, Agora does not recommend using it for accurate calculation.
Parameters
- imageWidth
- The width (px) of the video image captured by the local camera.
- imageHeight
- The height (px) of the video image captured by the local camera.
- vecRectangle
-
The information of the detected human face:
x
: The x-coordinate (px) of the human face in the local view. Taking the top left corner of the view as the origin, the x-coordinate represents the horizontal position of the human face relative to the origin.y
: The y-coordinate (px) of the human face in the local view. Taking the top left corner of the view as the origin, the y-coordinate represents the vertical position of the human face relative to the origin.width
: The width (px) of the human face in the captured view.height
: The height (px) of the human face in the captured view.
- vecDistance
- The distance between the human face and the device screen (cm).
- numFaces
- The number of faces detected. If the value is 0, it means that no human face is detected.
onFirstLocalVideoFrame
Occurs when the first local video frame is displayed on the local video view.
virtual void onFirstLocalVideoFrame(VIDEO_SOURCE_TYPE source, int width, int height, int elapsed) { (void)source; (void)width; (void)height; (void)elapsed; }
The SDK triggers this callback when the first local video frame is displayed on the local video view.
Parameters
- source
- The capture type of the custom video source. See VIDEO_SOURCE_TYPE.
- width
- The width (px) of the first local video frame.
- height
- The height (px) of the first local video frame.
- elapsed
- Time elapsed (ms) from the local user calling joinChannel [2/2] until the SDK triggers this callback. If you call startPreview [1/2] before calling joinChannel [2/2], then this parameter is the time elapsed from calling the startPreview [1/2] method until the SDK triggers this callback.
onFirstLocalVideoFramePublished
Occurs when the first video frame is published.
virtual void onFirstLocalVideoFramePublished(VIDEO_SOURCE_TYPE source, int elapsed) { (void)source; (void)elapsed; }
- The local client enables the video module and calls joinChannel [2/2] successfully.
- The local client calls muteLocalVideoStream(
true
) and muteLocalVideoStream(false
) in sequence. - The local client calls disableVideo and enableVideo in sequence.
- The local client calls pushVideoFrame to successfully push the video frame to the SDK.
Parameters
- source
- The capture type of the custom video source. See VIDEO_SOURCE_TYPE.
- elapsed
- Time elapsed (ms) from the local user calling joinChannel [2/2] until the SDK triggers this callback.
onFirstRemoteVideoDecoded
Occurs when the first remote video frame is received and decoded.
virtual void onFirstRemoteVideoDecoded(uid_t uid, int width, int height, int elapsed) { (void)uid; (void)width; (void)height; (void)elapsed; }
- The remote user joins the channel and sends the video stream.
- The remote user stops sending the video stream and re-sends it after 15 seconds. Reasons for such an interruption include:
- The remote user leaves the channel.
- The remote user drops offline.
- The remote user calls muteLocalVideoStream to stop sending the video stream.
- The remote user calls disableVideo to disable video.
Parameters
- uid
- The ID of the remote user sending the video stream.
- width
- The width (px) of the video stream.
- height
- The height (px) of the video stream.
- elapsed
- The time elapsed (ms) from the local user calling joinChannel [2/2] until the SDK triggers this callback.
onFirstRemoteVideoFrame
Occurs when the renderer receives the first frame of the remote video.
virtual void onFirstRemoteVideoFrame(uid_t userId, int width, int height, int elapsed) { (void)userId; (void)width; (void)height; (void)elapsed; }
Parameters
- userId
- The ID of the remote user sending the video stream.
- width
- The width (px) of the video stream.
- height
- The height (px) of the video stream.
- elapsed
- The time elapsed (ms) from the local user calling joinChannel [2/2] until the SDK triggers this callback.
onLocalVideoStateChanged
Occurs when the local video stream state changes.
virtual void onLocalVideoStateChanged(VIDEO_SOURCE_TYPE source, LOCAL_VIDEO_STREAM_STATE state, LOCAL_VIDEO_STREAM_ERROR error) { (void)source; (void)state; (void)error; }
When the state of the local video stream changes (including the state of the video capture and encoding), the SDK triggers this callback to report the current state. This callback indicates the state of the local video stream, including camera capturing and video encoding, and allows you to troubleshoot issues when exceptions occur.
LOCAL_VIDEO_STREAM_STATE_FAILED
and error code of LOCAL_VIDEO_STREAM_ERROR_CAPTURE_FAILURE
in the following situations:- The app switches to the background, and the system gets the camera resource.
- The camera starts normally, but does not output video frames for four consecutive seconds.
When the camera outputs the captured video frames, if the video frames are the same for 15 consecutive frames, the SDK triggers the onLocalVideoStateChanged callback with the state code of LOCAL_VIDEO_STREAM_STATE_CAPTURING and error code of LOCAL_VIDEO_STREAM_ERROR_CAPTURE_FAILURE. Note that the video frame duplication detection is only available for video frames with a resolution greater than 200 × 200, a frame rate greater than or equal to 10 fps, and a bitrate less than 20 Kbps.
Parameters
- source
- The capture type of the custom video source. See VIDEO_SOURCE_TYPE.
- state
-
The state of the local video, see LOCAL_VIDEO_STREAM_STATE.
- error
-
The detailed error information, see LOCAL_VIDEO_STREAM_ERROR.
onLocalVideoStats
Reports the statistics of the local video stream.
virtual void onLocalVideoStats(VIDEO_SOURCE_TYPE source, const LocalVideoStats& stats) { (void)source; (void)stats; }
The SDK triggers this callback once every two seconds to report the statistics of the local video stream.
Parameters
- source
- The capture type of the custom video source. See VIDEO_SOURCE_TYPE.
- stats
- The statistics of the local video stream. See LocalVideoStats.
onRemoteVideoStateChanged
Occurs when the remote video stream state changes.
virtual void onRemoteVideoStateChanged(uid_t uid, REMOTE_VIDEO_STATE state, REMOTE_VIDEO_STATE_REASON reason, int elapsed) { (void)uid; (void)state; (void)reason; (void)elapsed; }
Parameters
- uid
- The ID of the remote user whose video state changes.
- state
-
The state of the remote video, see REMOTE_VIDEO_STATE.
- reason
-
The reason for the remote video state change, see REMOTE_VIDEO_STATE_REASON.
- elapsed
- Time elapsed (ms) from the local user calling the joinChannel [2/2] method until the SDK triggers this callback.
onRemoteVideoStats
Reports the statistics of the video stream sent by each remote users.
virtual void onRemoteVideoStats(const RemoteVideoStats& stats) { (void)stats; }
Reports the statistics of the video stream from the remote users. The SDK triggers this callback once every two seconds for each remote user. If a channel has multiple users/hosts sending video streams, the SDK triggers this callback as many times.
Parameters
- stats
- Statistics of the remote video stream. For details, see RemoteVideoStats.
onRemoteVideoTransportStats
Reports the transport-layer statistics of each remote video stream.
virtual void onRemoteVideoTransportStats(uid_t uid, unsigned short delay, unsigned short lost, unsigned short rxKBitRate) { (void)uid; (void)delay; (void)lost; (void)rxKBitRate; }
- Deprecated:
- This callback is deprecated; use onRemoteVideoStats instead.
This callback reports the transport-layer statistics, such as the packet loss rate and network time delay, once every two seconds after the local user receives a video packet from a remote user.
During a call, when the user receives the video packet sent by the remote user/host, the callback is triggered every 2 seconds.
Parameters
- uid
- The ID of the remote user sending the video packets.
- delay
- The network delay (ms) from the sender to the receiver.
- lost
- The packet loss rate (%) of the video packet sent from the remote user.
- rxKBitRate
- The bitrate of the received video (Kbps).
onSnapshotTaken
Reports the result of taking a video snapshot.
virtual void onSnapshotTaken(uid_t uid, const char* filePath, int width, int height, int errCode) { (void)uid; (void)filePath; (void)width; (void)height; (void)errCode; }
After a successful takeSnapshot method call, the SDK triggers this callback to report whether the snapshot is successfully taken as well as the details for the snapshot taken.
Parameters
- uid
- The user ID. One uid of 0 indicates the local user.
- filePath
- The local path of the snapshot.
- width
- The width (px) of the snapshot.
- height
- The height (px) of the snapshot.
- errCode
- The message that confirms success or gives the reason why the snapshot is not successfully taken:
- 0: Success.
- < 0: Failure:
- -1: The SDK fails to write data to a file or encode a JPEG image.
- -2: The SDK does not find the video stream of the specified user within one second after the takeSnapshot method call succeeds. The possible reasons are: local capture stops, remote end stops publishing, or video data processing is blocked.
- -3: Calling the takeSnapshot method too frequently.
onUserEnableLocalVideo
Occurs when a specific remote user enables/disables the local video capturing function.
virtual void onUserEnableLocalVideo(uid_t uid, bool enabled) { (void)uid; (void)enabled; }
The SDK triggers this callback when the remote user resumes or stops capturing the video stream by calling the enableLocalVideo method.
Parameters
- uid
- The user ID of the remote user.
- enabled
-
Whether the specified remote user enables/disables the local video capturing function:
true
: Enable. Other users in the channel can see the video of this remote user.false
: Disable. Other users in the channel can no longer receive the video stream from this remote user, while this remote user can still receive the video streams from other users.
onUserEnableVideo
Occurs when a remote user enables/disables the video module.
virtual void onUserEnableVideo(uid_t uid, bool enabled) { (void)uid; (void)enabled; }
Once the video module is disabled, the user can only use a voice call. The user cannot send or receive any video.
The SDK triggers this callback when a remote user enables or disables the video module by calling the enableVideo or disableVideo method.
Parameters
- uid
- The user ID of the remote user.
- enabled
-
true
: Enable.false
: Disable.
onUserMuteVideo
Occurs when a remote user stops/resumes publishing the video stream.
virtual void onUserMuteVideo(uid_t uid, bool muted) { (void)uid; (void)muted; }
When a remote user calls muteLocalVideoStream to stop or resume publishing the video stream, the SDK triggers this callback to report the state of the remote user's publishing stream to the local user.
Parameters
- uid
- The user ID of the remote user.
- muted
- Whether the remote user stops publishing the video stream:
true
: The remote user stops publishing the video stream.false
: The remote user resumes publishing the video stream.
onVideoDeviceStateChanged
Occurs when the video device state changes.
virtual void onVideoDeviceStateChanged(const char* deviceId, int deviceType, int deviceState) { (void)deviceId; (void)deviceType; (void)deviceState; }
This callback reports the change of system video devices, such as being unplugged or removed. On a Windows device with an external camera for video capturing, the video disables once the external camera is unplugged.
Parameters
- deviceId
- The device ID.
- deviceType
- Media device types. See MEDIA_DEVICE_TYPE.
- deviceState
- Media device states.
onVideoPublishStateChanged
Occurs when the video publishing state changes.
virtual void onVideoPublishStateChanged(VIDEO_SOURCE_TYPE source, const char* channel, STREAM_PUBLISH_STATE oldState, STREAM_PUBLISH_STATE newState, int elapseSinceLastState) { (void)source; (void)channel; (void)oldState; (void)newState; (void)elapseSinceLastState; }
Parameters
- channel
- The channel name.
- source
- The capture type of the custom video source. See VIDEO_SOURCE_TYPE.
- oldState
- For the previous publishing state, see STREAM_PUBLISH_STATE.
- newState
- For the current publishing state, see STREAM_PUBLISH_STATE.
- elapseSinceLastState
- The time elapsed (ms) from the previous state to the current state.
onVideoSizeChanged
Occurs when the video size or rotation of a specified user changes.
virtual void onVideoSizeChanged(uid_t uid, int width, int height, int rotation) { (void)uid; (void)width; (void)height; (void)rotation; }
Parameters
- uid
- The ID of the user whose video size or rotation changes. (The uid for the local user is 0. The video is the local user's video preview).
- width
- The width (pixels) of the video stream.
- height
- The height (pixels) of the video stream.
- rotation
- The rotation information. The value range is [0,360).
onVideoStopped
Occurs when the video stops playing.
virtual void onVideoStopped()
- Deprecated:
- Use LOCAL_VIDEO_STREAM_STATE_STOPPED(0) in the onLocalVideoStateChanged callback instead.
The application can use this callback to change the configuration of the view (for example, displaying other pictures in the view) after the video stops playing.
onVideoSubscribeStateChanged
Occurs when the video subscribing state changes.
virtual void onVideoSubscribeStateChanged(const char* channel, uid_t uid, STREAM_SUBSCRIBE_STATE oldState, STREAM_SUBSCRIBE_STATE newState, int elapseSinceLastState) { (void)channel; (void)uid; (void)oldState; (void)newState; (void)elapseSinceLastState; }
Parameters
- channel
- The channel name.
- uid
- The ID of the remote user.
- oldState
- The previous subscribing status, see STREAM_SUBSCRIBE_STATE for details.
- newState
- The current subscribing status, see STREAM_SUBSCRIBE_STATE for details.
- elapseSinceLastState
- The time elapsed (ms) from the previous state to the current state.