Use an Extension

Extensions are add-ons designed to extend the functionality of the Agora SDK. The Extensions Marketplace is home to extensions that make your app more fun. These extensions provide features such as:

• Audio effects and voice changing.
• Face filters and background removal.
• Live transcription and captioning.

This page shows you how to integrate and use an extension from the Agora Extensions Marketplace.

Understand the tech

To use an extension in your Agora project, implement the following steps in your app:

1. Add the extension to your app: Call addExtension when you initialize RtcEngine.
2. Enable the extension: Call enableExtension before joining a channel.
3. Use an extension feature: Call setExtensionProperty to pass in necessary parameters.

Prerequisites

To follow the procedure on this page, you must have:

• An Agora project that has successfully started Video Call or Interactive Live Streaming Premium.
• The Agora SDK v4.0.0 Preview. Currently only this SDK supports extensions.
• An extension package (.aar or .so) and details about the extension. See Activate an extension.

Project setup

To create the environment necessary to integrate the extension into your app, do the following:

• Android Archive file (.aar)
  1. Save the .aar file to /app/libs in your project folder.
  2. In /Gradle Scripts/build.gradle(Module: <ProjectName> app), add the following line under dependencies:

     implementation fileTree(include: ['*.jar', '*.aar'], dir: 'libs')

• Shared Library (.so)
  Save the .so file to the following paths in your project folder:
  • /app/src/main/jniLibs/arm64-v8a
  • /app/src/main/jniLibs/armeabi-v7a

Implement the extension in your project

For demonstration purposes, a simple watermark extension is used to explain the procedure.

The watermark extension adds a watermark on video streamed to your local client. This section shows you how to implement the watermark extension in your Agora project.

Import necessary classes

1. Download the watermark extension and follow the steps for .aar files in Project setup.
2. In app/src/main/java/com.example.<projectname>/MainActivity:
   1. Add the following lines to import the Android classes used by the extension:

      import org.json.JSONException;
      import org.json.JSONObject;

   2. Add the following lines to import the Agora classes used by the extension:

      // ExtensionManager is used to pass in basic information about the extension
      import io.agora.extension.ExtensionManager;
      import io.agora.rtc2.IMediaExtensionObserver;
      import io.agora.rtc2.RtcEngineConfig;

Enable and use the extension

In app/src/main/java/com.example.<projectname>/MainActivity, replace the current initializeAndJoinChannel function with the following code:

private void initializeAndJoinChannel() {
    try {
        RtcEngineConfig config = new RtcEngineConfig();
        config.mContext = getBaseContext();
        config.mAppId = appId;
        // Call addExtension to add the extension.
        config.addExtension(ExtensionManager.EXTENSION_NAME);
        // Register IMediaExtensionObserver to receive events from the extension.
        config.mExtensionObserver = new IMediaExtensionObserver() {
            @Override
            public void onEvent(String vendor, String extension, String key, String value) {
                // Add callback handling logics for extension events.
            }
        };
        config.mEventHandler = mRtcEventHandler;
        mRtcEngine = RtcEngine.create(config);
    } catch (Exception e) {
        throw new RuntimeException("Check the error.");
    }

    // Call enableExtension to enable the extension.
    // To enable multiple extensions, call enableExtension as many times. The sequence of enabling multiple extensions determines the order of these extensions in the transmission pipeline.
    // For example, if you enable extension A before extension B, extension A processes data from the SDK before extension B.
    mRtcEngine.enableExtension(ExtensionManager.EXTENSION_VENDOR_NAME, ExtensionManager.EXTENSION_VIDEO_FILTER_NAME, true);

    mRtcEngine.enableVideo();
    mRtcEngine.startPreview();
    FrameLayout container = findViewById(R.id.local_video_view_container);
    SurfaceView surfaceView = new SurfaceView(getBaseContext());
    container.addView(surfaceView);
    mRtcEngine.setupLocalVideo(new VideoCanvas(surfaceView, VideoCanvas.RENDER_MODE_FIT, 0));

    ChannelMediaOptions options = new ChannelMediaOptions();
    options.clientRoleType = Constants.CLIENT_ROLE_BROADCASTER;
    options.channelProfile = Constants.CHANNEL_PROFILE_LIVE_BROADCASTING;
    options.publishCameraTrack = true;

    mRtcEngine.joinChannel(token, channelName, 0, options);

    JSONObject o = new JSONObject();
    try {
        // Pass in the key-value pairs defined by the extension provider to configure the feature you want to use.
        o.put("plugin.watermark.wmStr", "Agora");
        o.put("plugin.watermark.wmEffectEnabled", true);

        // Call setExtensionProperty to use the watermark feature.
        mRtcEngine.setExtensionProperty(ExtensionManager.EXTENSION_VENDOR_NAME, ExtensionManager.EXTENSION_VIDEO_FILTER_NAME, "key", o.toString());
    } catch (JSONException e) {
        e.printStackTrace();
    }
}

Test the extension

To test whether the extension works properly, follow these steps:

1. Connect the Android device to the computer.
2. Click Run app on your Android Studio. A moment later you will see the project installed on your device.
3. When the app launches, you can see yourself and the watermark "Agora" on the local view.

Sample project

Agora provides an open-source sample project agora-simple-filter on GitHub for your reference.