By integrating Banuba Face AR SDK and Agora.io SDK you can enable augmented reality features in video calls such as face filters, face touch up filters and virtual backgrounds. You can find the integration examples below.
To receive a trial token or a full commercial licence from Banuba - please fill in our form on form on banuba.com website, or contact us via info@banuba.com.
Get the following Banuba trial client token.
To receive a trial token or a full commercial licence from Banuba - please fill in our form on form on banuba.com website, or contact us via info@banuba.com.
Execute pod install
to get the Banuba SDK.
Open the BanubaAgoraFilters.xcworkspace
file in Xcode.
Copy and Paste your Banuba client token into the appropriate section of /BanubaAgoraFilters/Token.swift
with “ ” symbols. For example:
let banubaClientToken = "Banuba Token"
Visit agora.io to sign up and get the token, app and channel ID.
Copy and Paste your agora token, app and chanel ID into appropriate section of /BanubaAgoraFilters/Token.swift
with “ ” symbols. For example:
internal let agoraAppID = "Agora App ID"
internal let agoraClientToken = "Agora Token"
internal let agoraChannelId = "Agora Channel ID"
Donwload the needed effects from here. This guarantees, that you will use the up-to-date version of the effects. The effects must be copied to the agora-plugin-filters-ios -> BanubaAgoraFilters -> effects
folder.
Open the BanubaAgoraFilters.xcodporj
project in Xcode and run the BanubaAgoraFilters
target.
Connecting Banuba SDK to your project is similar to the steps in the Getting Started section. As for AgoraRtcKit, we advise to use Swift Package Manager. You should use the following settings:
URL: https://github.com/agorabuilder/AgoraRtcEngine_iOS_Preview.git
Version Rule: Exact
Version: 4.0.0-preview.6
BanubaFiltersAgoraExtension
To control BanubaFiltersAgoraExtension
with Agora libs look available keys listed below:
public struct BanubaPluginKeys {
public static let vendorName = "Banuba"
public static let extensionName = "BanubaFilter"
public static let loadEffect = "load_effect"
public static let unloadEffect = "unload_effect"
public static let setEffectsPath = "set_effects_path"
public static let setToken = "set_token"
}
To enable/disable BanubaFiltersAgoraExtension
use the following method:
import BanubaFiltersAgoraExtension
agoraKit?.enableExtension(
withVendor: BanubaPluginKeys.vendorName,
extension: BanubaPluginKeys.extensionName,
enabled: true
)
Before applying an effect on your video you have to initialize BanubaFiltersAgoraExtension
with the path to effects and banuba client token. Look how it can be achieved:
agoraKit?.setExtensionPropertyWithVendor(
BanubaPluginKeys.vendorName,
extension: BanubaPluginKeys.extensionName,
key: BanubaPluginKeys.setEffectsPath,
value: "place_path_to_effects_folder_here"
)
agoraKit?.setExtensionPropertyWithVendor(
BanubaPluginKeys.vendorName,
extension: BanubaPluginKeys.extensionName,
key: BanubaPluginKeys.setToken,
value: "place_your_banuba_token_here".trimmingCharacters(in: .whitespacesAndNewlines)
)
After those steps you can tell BanubaFiltersAgoraExtension
to enable or disable the mask:
agoraKit?.setExtensionPropertyWithVendor(
BanubaPluginKeys.vendorName,
extension: BanubaPluginKeys.extensionName,
key: BanubaPluginKeys.loadEffect,
value: "put_effect_name_here"
)
agoraKit?.setExtensionPropertyWithVendor(
BanubaPluginKeys.vendorName,
extension: BanubaPluginKeys.extensionName,
key: BanubaPluginKeys.unloadEffect,
value: " "
)
If the mask has parameters and you want to change them, you can do it the next way:
agoraKit?.setExtensionPropertyWithVendor(
BanubaPluginKeys.vendorName,
extension: BanubaPluginKeys.extensionName,
key: BanubaPluginKeys.evalJSMethod,
value: string
)
string
must be a string with method’s name and parameters. You can find an example in our documentation.
BanubaFiltersAgoraExtension
To build the BanubaFiltersAgoraExtension
manually, please follow the steps bellow:
Execute pod install
to get the Banuba SDK.
Open the BanubaAgoraFilters.xcworkspace
file in Xcode.
Choose File->Packages->Reset Package Cashes from Xcode menu.
Build the target BanubaFiltersAgoraExtension
. It will be built with your Swift version. After this you should open the section Products in the Project Navigator (the left part of the Xcode screen). Click on the BanubaFiltersAgoraExtension
with the right click of the mouse and choose Show in Finder. Copy the BanubaFiltersAgoraExtension.framework
from the folder.
Then put the framework to the /Frameworks
folder of the BanubaAgoraFilters.xcodeproj
(or of your project). Then you can build BanubaAgoraFilters or your project.
The reconnection of the BanubaFiltersAgoraExtension.framework
to the example project may be required. To do it, you should remove the BanubaFiltersAgoraExtension.framework
from the Project Settings: General-> Frameworks, Libraries and Embedded Content. Then you should drag&drop the BanubaFiltersAgoraExtension.framework
to this section. You should choose Embed&Sign for this framework.
To retrieve effects list use the following code:
let effectsPath = BanubaEffectsManager.effectsURL.path
let effectsService = EffectsService(effectsPath: effectsPath)
let effectViewModels =
effectsService
.getEffectNames()
.sorted()
.compactMap { effectName in
guard let effectPreviewImage = effectsService.getEffectPreview(effectName) else {
return nil
}
let effectViewModel = EffectViewModel(image: effectPreviewImage, effectName: effectName)
return effectViewModel
}
EffectViewModel
has the following properties:
class EffectViewModel {
let image: UIImage
let effectName: String?
var cancelEffectModel: Bool {
return effectName == nil
}
}
.aar
file from the Banuba SDK archive into libs
dir:
banuba_effect_player-release.aar
=>libs/
include/bnb
directory into libs/bnb_sdk/
:
include/bnb
=>libs/bnb_sdk/bnb
jar
files from the Agora Video SDK archive into libs
dir:
agora-rtc-sdk.jar
=>libs/
jniLibs
dir:
arm64-v8a
,armeabi-v7a
andx86_64
=>libs/jniLibs/
*.h
files) into libs/agora_rtc_sdk/include
:
low_level_api/include
=>libs/agora_rtc_sdk/include
app/src/main/java/com/banuba/sdk/agorapluginexample/ClientToken.kt
with " " symbols. For example: BANUBA_CLIENT_TOKEN = "place_your_token_here"app/src/main/java/com/banuba/sdk/agorapluginexample/ClientToken.kt
with " " symbols. For example: AGORA_APP_ID = "place_your_token_here"BanubaFiltersAgoraExtension
Add following imports:
import com.banuba.agora.plugin.BanubaEffectsLoader
import com.banuba.agora.plugin.BanubaExtensionManager
import com.banuba.agora.plugin.BanubaResourceManager
import com.banuba.agora.plugin.model.ArEffect
import com.banuba.sdk.utils.ContextProvider
Add extension to RtcEngineConfig:
RtcEngineConfig().apply {
...
System.loadLibrary("banuba")
addExtension(BanubaExtensionManager.EXTENSION_NAME)
ContextProvider.setContext(mContext)
...
}
Create and initialize BanubaResourceManager:
private val banubaResourceManager by lazy(LazyThreadSafetyMode.NONE) {
BanubaResourceManager(this)
}
override fun onCreate(savedInstanceState: Bundle?) {
...
banubaResourceManager.prepare()
}
After those steps enable and initialize extension:
agoraRtc.enableExtension(
BanubaExtensionManager.VENDOR_NAME,
BanubaExtensionManager.VIDEO_FILTER_NAME,
true
)
private fun initBanubaPlugin() {
agoraRtc.setExtensionProperty(
BanubaExtensionManager.VENDOR_NAME,
BanubaExtensionManager.VIDEO_FILTER_NAME,
BanubaExtensionManager.KEY_SET_RESOURCES_PATH,
banubaResourceManager.resourcesPath
)
agoraRtc.setExtensionProperty(
BanubaExtensionManager.VENDOR_NAME,
BanubaExtensionManager.VIDEO_FILTER_NAME,
BanubaExtensionManager.KEY_SET_EFFECTS_PATH,
banubaResourceManager.effectsPath
)
agoraRtc.setExtensionProperty(
BanubaExtensionManager.VENDOR_NAME,
BanubaExtensionManager.VIDEO_FILTER_NAME,
BanubaExtensionManager.KEY_SET_TOKEN,
BANUBA_CLIENT_TOKEN
)
}
To enable/disable effects use the following code:
private val onEffectPrepared = object : BanubaResourceManager.EffectPreparedCallback {
override fun onPrepared(effectName: String) {
sendEffectToFilter(effectName)
}
}
private fun sendEffectToFilter(effect: String) {
agoraRtc.setExtensionProperty(
BanubaExtensionManager.VENDOR_NAME,
BanubaExtensionManager.VIDEO_FILTER_NAME,
BanubaExtensionManager.KEY_LOAD_EFFECT,
effect
)
}
banubaResourceManager.prepareEffect(Effect name, onEffectPrepared)
To retrieve current effects use the following code:
val effects: List<ArEffect> = BanubaEffectsLoader(this).loadEffects()
ArEffect contains following information:name: String
- pass to banubaResourceManager.prepareEffect(Effect name, onEffectPrepared)
. Also can be used to display label on the UIpreview: Bitmap
- can be used as preview image
To modify effects, add or remove effect folder in app/src/main/assets/effects
directory.
By default sample contains the following effects:
ElvisUnleashed
EnglandEightPieceCap
FashionHistory1940_male
MorphingHatBow
Nerd
SnapBubblesGlasses
Space
StarGlow
TitanicJack
Android:
iOS: