The image enhancement extension is used together with the Agora Web SDK (v4.12.0 or later). This extension gives you granular control over the degree of image enhancement, such as skin lightening, skin smoothing, and red saturation.
The following figures show how the extension touches up the image:
The browser support for the image enhancement extension is as follows:
The image enhancement extension has high performance requirements. Make sure your computer has the following hardware and software:
When using the image enhancement extension, ensure that hardware acceleration is enabled in the browser.
The image enhancement extension incorporates and upgrades the built-in image enhancement feature (enabled by setBeautyEffect) of the Agora Web SDK 4.x. Agora is planning to sunset this built-in image enhancement feature. If you have implemented this image enhancement feature in your app, Agora recommends upgrading to v4.12.0 or later and using the image enhancement extension as a replacement.
If you use multiple media processing extensions at the same time, Agora recommends an Intel i5 quad-core CPU or better. When multiple extensions are enabled, if another app is consuming more system resources, the audio and video might freeze in your app.
A typical transmission pipeline in the Agora Web SDK consists of a chain of procedures, including capture, preprocessing, encoding, transmitting, decoding, post-processing, and playback. In the preprocessing stage, extensions can modify the audio and video data in the pipeline to implement features such as virtual background, noise cancellation, and image enhancement.
To integrate and implement the image enhancement extension, follow these steps:
Refer to the appropriate Quickstart Guide to integrate the Web SDK (v4.12.0 or later) and implement the basic real-time communication functions in your project.
Integrate the image enhancement extension into your project via npm:
To install the extension, run the following command:
npm install agora-extension-beauty-effect
To import the extension, use either of the following methods:
Method one: Add the following code to the JavaScript file:
import BeautyExtension from "agora-extension-beauty-effect";
Method two: Use the Script tag in the HTML file. Once imported, the BeautyExtension
instance can be used directly in JavaScript files.
<script src="./agora-extension-beauty-effect.js"></script>
Register the extension: After calling AgoraRTC.createClient
to create a client object, create a new BeautyExtension
instance and pass in the BeautyExtension
instance when calling AgoraRTC.registerExtensions
.
// Create a client object
var client = AgoraRTC.createClient({mode: "rtc", codec: "vp8"});
// Create a BeautyExtension instance
const extension = new BeautyExtension();
// Register the extension
AgoraRTC.registerExtensions([extension]);
Call extension.createProcessor
to create
a BeautyProcessor
instance.
const processor = extension.createProcessor();
After creating a local camera track, call videoTrack.pipe
and specify videoTrack.processorDestination
to inject the extension into the SDK's media processing pipeline.
localTracks.videoTrack.pipe(processor).pipe(localTracks.videoTrack.processorDestination);
Call processor.enable
to enable the image enhancement extension.
await processor.enable();
After calling this method, the SDK uses the default value of each parameter in BeautyEffectOptions.
Call processor.setOptions
to adjust the image enhancement parameters.
processor.setOptions({
// Set the contrast level as high (2)
lighteningContrastLevel: 2,
// Set the brightness level
lighteningLevel: 0.7,
// Set the smoothness level
smoothnessLevel: 0.6,
// Set the sharpness level
sharpnessLevel: 0.5,
// Set the redness level
rednessLevel: 0.5
});
import AgoraRTC from "agora-rtc-sdk-ng";
import BeautyExtension from "agora-extension-beauty-effect";
// Create a client
var client = AgoraRTC.createClient({mode: "rtc", codec: "vp8"});
// Create a BeautyExtension instance
const extension = new BeautyExtension();
// Register the extension
AgoraRTC.registerExtensions([extension]);
// Create a BeautyProcessor instance
const processor = extension .createProcessor();
var localTracks = {
videoTrack: null,
audioTrack: null
};
async function start() {
// Create a local microphone track and camera track
localTracks.audioTrack = localTracks.audioTrack ||
await AgoraRTC.createMicrophoneAudioTrack();
localTracks.videoTrack = localTracks.videoTrack ||
await AgoraRTC.createCameraVideoTrack({cameraId: videoSelect.value, encoderConfig: '720p_2'});
localTracks.videoTrack.play("local-player");
if (processor && localTracks.videoTrack) {
// Inject the extension into the video processing pipeline in the SDK
localTracks.videoTrack.pipe(processor).pipe(localTracks.videoTrack.processorDestination);
// Enable image enhancement
await processor.enable();
}
}
// Adjust image enhancement parameters
async function setBeautyOptions() {
processor.setOptions({
lighteningContrastLevel: 2,
lighteningLevel: 0.7,
smoothnessLevel: 0.6,
sharpnessLevel: 0.5,
rednessLevel: 0.5
});
}
createProcessor
createProcessor(): IBeautyProcessor;
Creates an IBeautyProcessor
instance.
setOptions(options: BeautyEffectOptions): void;
Sets image enhancement parameters.
Parameters:
options
: The image enhancement parameters. See BeautyEffectOptions for details.enable(): void | Promise<void>;
Enables the image enhancement extension.
If you do not call setOptions
before calling this method, the SDK uses the default value of each parameter in BeautyEffectOptions.
disable(): void | Promise<void>;
Disables the image enhancement extension.
onoverload?: () => void;
When the system performance cannot meet the processing requirements, the SDK triggers onoverload
.
Agora recommends calling disable
in this event to disable image enhancement and adding a UI prompt.
export type BeautyEffectOptions = {
lighteningContrastLevel: 0 | 1 | 2;
lighteningLevel: Number,
smoothnessLevel?: Number;
sharpnessLevel?: Number;
rednessLevel?: Number;
};
The image enhancement parameters. Used in the setOptions method.
lighteningContrastLevel
: The contrast level, used with the lighteningLevel
parameter. The larger the value, the greater the contrast between light and dark. Possible values include:
0
: Low contrast.1
: (Default) Normal contrast.2
: High contrast.lighteningLevel
: Number. The brightness level, in the range [0.0,1.0], where 0.0 means the original brightness. The default value is 0.6. This parameter is used to lighten up a user's skin. The larger the value, the greater the lightning effect.
smoothnessLevel
: Number. The smoothness level, in the range [0.0,1.0], where 0.0 means the original smoothness. The default value is 0.5. This parameter is used to remove blemishes and smooth a user's skin. The larger the value, the greater the skin smoothing effect.
sharpnessLevel
: Number. The sharpness level, in the range [0.0,1.0], where 0.0 means the original sharpness. The default value is 0.3. This parameter is used to improve the sharpness level of the image. The larger the value, the greater the sharpening effect.
rednessLevel
: Number. The redness level, in the range [0.0,1.0], where 0.0 means the original redness. The default value is 0.1. This parameter adjusts the red saturation of the image. The larger the value, the greater the red saturation effect.