Start Content Moderation

This page describes how to implement Agora Content Moderation in your Android, iOS, macOS, Windows apps.

Understand the tech

The following figure shows the basic workflow of integrating content moderation in the app:

The process of applying real-time content moderation to the video of a user streaming in a channel is as follows:

1. After calling enableContentInspect to enable content moderation, the content moderation module of the SDK takes a screenshot of the video stream and inspects it. If the inspection result of the picture is confirmed, the SDK triggers the onContentInspectResult callback and returns the content moderation results: neutral, sexy, or porn.
2. After the content moderation is completed, the SDK uploads the image to the Agora Content Moderation cloud server. If the SDK inspection result of the image is inconclusive, the Agora Content Moderation service conducts a second inspection of the image.
3. After the content moderation process for an image is complete, the content moderation service sends the result in the form of an HTTP request to the server address you specify, and the SDK triggers the onContentInspectResult callback and returns the inspection result.

Prerequisites

Before you start using Agora Content Moderationensure that the following requirements are met:

• Contact sales@agora.io to enable Agora Content Moderation service. The following elements must be provided in accordance with the format:

  "AppId": {
    "desc": "Description",
    "score_threshold": [
        {"scene": "porn or sexy or neutral", "threshold": "Judgment threshold, which range is [0,1]"}  // Set the threshold of different types, support setting up to 2 thresholds
    ],
    "moderation_upload_policy":  ["porn", "sexy", "neutral"],   // Set the types of the images to be stored, if not set, the porn and sexy images are stored by default
    "callback_url": "The address of the server that receives the content moderation result",
    "callback_url_supervisor": "The address of the server that receives the screenshot callback",
    "aws_oss_access_key_id": "AWS-OSS-AccessKeyId",
    "aws_oss_access_key_secret": "AWS-OSS-AccessKeySecret",
    "aws_oss_bucket_name": "AWS-OSS-BucketName",
    "aws_oss_endpoint": "AWS-OSS-Endpoint",
    "aws_oss_region": "AWS-OSS-region",
    "aws_oss_fileName_prefix": "The storage location of the screenshots in AWS OSS"
  },

• Ensure that Agora Video SDK of version 3.6.2 or later has been integrated into your project (including the Content Moderation plug-in), join the channel, and publish a video stream. For details, see Get Started with Interactive Live Streaming Premium.

Implement Content Moderation

Enable Content Moderation

Call the enableContentInspect method to enable the content moderation. Set the type in ContentInspectConfig to MODERATION (1), and call enableContentInspect to enable the content moderation. The SDK takes screenshots according to the set frequency and uploads the screenshots to the Agora Content Moderation cloud server. Sample code:

Android

ContentInspectConfig config = new ContentInspectConfig();
config.moduleCount = 1;
// The type of the function module is content moderation.
config.modules[0].type = ContentInspectConfig.CONTENT_INSPECT_TYPE_MODERATION;
// The frequency of content moderation is 2 seconds between image captures.
config.modules[0].interval = 2;
mRtcEngine.enableContentInspect(true, config);

iOS/macOS

let inspectExtraConfig = AgoraContentInspectConfig()
var modules = [AgoraContentInspectModule]()
let module1 = AgoraContentInspectModule()
// The type of the function module is content moderation.
module1.type = .moderation
// The frequency of content moderation is 2 seconds between image captures.
module1.interval = 2
modules.append(module1)
inspectExtraConfig.modules = modules
agoraKit.enableContentInspect(true, config:inspectExtraConfig)

Windows

void CMainFrame::OnBtnEnableContentInspect()
{
    if (CAgoraObject::GetEngine() == NULL) {
        return;
    }

        ContentInspectConfig config;
        ContentInspectModule module1;
        // The type of the function module is content moderation.
        module1.type = kContentInspectModeration;
        // The frequency of content moderation is 2 seconds between image captures.
        module1.interval = 2;

        config.moduleCount = 1;
        config.modules[0] = module1;

        CAgoraObject::GetEngine()->enableContentInspect(true, config);
}

Monitor the content moderation callback

After the client-side content moderation is completed, the SDK triggers the onContentInspectResult callback and returns the inspection result. Refer to the following sample code to monitor the content moderation callback.

Android

public void onContentInspectResult(int result) {
}

iOS/macOS

// OC
(void)rtcEngine:(AgoraRtcEngineKit* _Nonnull)engine contentInspectResult:(AgoraContentInspectResult)result{
}
// Swift
func rtcEngine(_ engine: AgoraRtcEngineKit, contentInspectResult result: AgoraContentInspectResult) {
}

Windows

void onContentInspectResult(CONTENT_INSPECT_RESULT result){
}

Related Information

API reference

Android

• enableContentInspect
• onContentInspectResult

iOS/macOS

• enableContentInspect
• contentInspectResult

Windows

• enableContentInspect
• onContentInspectResult

  Dynamic library of content moderation

The following table shows the dynamic library name for each platform and the increase in app size after integration: