Use this guide to quickly start the interactive live video streaming with the Agora Video SDK for Unity.
The following figure shows the workflow to integrate into your app in order to add Interactive Live Streaming Premium functionality.
As shown in the figure, the workflow for adding Interactive Live Streaming Premium in your project is as follows:
Set the client role
Each user in an Interactive Live Streaming Premium channel is either a host or an audience member. Hosts publish streams to the channel, and the audience subscribe to the streams.
Retrieve a token
A token is the credential that authenticates a user when your app client joins a channel. In a test or production environment, your app client retrieves tokens from a server in your security infrastructure.
Join a channel
Call joinChannel
to create and join a channel. App clients that pass the same channel name join the same channel.
Publish and subscribe to audio and video in the channel
After joining a channel, app clients with the role of the host can publish audio and video. For an auidence memeber to send audio and video, you can call setClientRole
to switch the client role.
For an app client to join a channel, you need the following information:
Unity 2017 or later
Operating system and IDE requirements:
Target Platform | Operating system version | IDE version |
---|---|---|
Android | Android 4.1 or later | Android Studio 3.0 or later |
iOS | iOS 9.0 or later | Xcode 9.0 or later |
macOS | macOS 10.10 or later | Xcode 9.0 or later |
Windows | Windows 7 or later | Microsoft Visual Studio 2017 or later |
A valid Agora account and an App ID
In this section, we create a Unity project and integrate the SDK into the project.
Use the following steps or follow the Unity official manual to build a project from scratch. Skip to Integrate the SDK if you have already created a Unity project.
Ensure that you have downloaded and installed Unity before the following steps. If not, click here to download.
Open Unity and click New.
Fill in and select the options in the following fields, and click Create project.
Choose either of the following approaches to integrate the Agora Unity SDK into your project.
Approach 1: Automatically integrate the SDK with Unity Asset Store
Approach 2: Manually add the SDK files
Go to SDK Downloads, download the Agora SDK for Unity under Video SDK, and extract the files from the downloaded SDK package.
Copy the Plugins
subfolder from the samples/Hello-Video-Unity-Agora/Assets/AgoraEngine
directory of the downloaded SDK to the Assets
subfolder of your project.
BL_BuildPostProcess.cs
file from the samples/Hello-Video-Unity-Agora/Assets/AgoraEngine/Editor
directory.This section provides instructions on using the Agora Video SDK to start the interactive live video streaming, as well as an API call sequence diagram.
Create the user interface (UI) for the interactive video streaming in your project. If you already have one UI in your project, skip to Get the device permission (Android only) or Initialize IRtcEngine.
We recommend adding the following elements to the UI:
If you use the Unity Editor to build your UI, ensure that you bind VideoSurface.cs to the GameObjects designated for local and remote videos.
If you build for Android, you should add in APIs to check and request the device permission. For all other platforms, this is handled by the engine, and you can skip to Initialize IRtcEngine.
Unity versions later than UNITY_2018_3_OR_NEWER do not automatically request camera and microphone permissions from your Android device. If you are using one of these versions, call the CheckPermission
method to request access to the camera and microphone of your Android device.
#if(UNITY_2018_3_OR_NEWER)
using UnityEngine.Android;
#endif
void Start ()
{
#if(UNITY_2018_3_OR_NEWER)
permissionList.Add(Permission.Microphone);
permissionList.Add(Permission.Camera);
#endif
}
private void CheckPermission()
{
#if(UNITY_2018_3_OR_NEWER)
foreach(string permission in permissionList)
{
if (Permission.HasUserAuthorizedPermission(permission))
{
}
else
{
Permission.RequestUserPermission(permission);
}
}
#endif
}
void Update ()
{
#if(UNITY_2018_3_OR_NEWER)
// Ask for your Android device's permissions.
CheckPermission();
#endif
}
Initialize the IRtcEngine
object before calling any other Agora APIs.
Call the GetEngine
method and pass in the App ID to initialize the IRtcEngine object.
Listen for callback events, such as when the local user joins the channel, and when the first video frame of a remote user is decoded.
// Pass an App ID to create and initialize an IRtcEngine object.
mRtcEngine = IRtcEngine.GetEngine (appId);
// Listen for the OnJoinChannelSuccessHandler callback.
// This callback occurs when the local user successfully joins the channel.
mRtcEngine.OnJoinChannelSuccessHandler = OnJoinChannelSuccessHandler;
// Listen for the OnUserJoinedHandler callback.
// This callback occurs when the first video frame of a remote user is received and decoded after the remote user successfully joins the channel.
// You can call the SetForUser method in this callback to set the remote video.
mRtcEngine.OnUserJoinedHandler = OnUserJoinedHandler;
// Listen for the OnUserOfflineHandler callback.
// This callback occurs when the remote user leaves the channel or drops offline.
mRtcEngine.OnUserOfflineHandler = OnUserOfflineHandler;
After initializing the IRtcEngine
object, call the SetChannelProfile
method to set the channel profile as LIVE_BROADCASTING
.
One IRtcEngine
object uses one profile only. If you want to switch to another profile, destroy the current IRtcEngine
object with the Destroy
method and create a new one before calling the SetChannelProfile
method.
// Set the channel profile as LIVE_BROADCASTING.
mRtcEngine.SetChannelProfile(CHANNEL_PROFILE.CHANNEL_PROFILE_LIVE_BROADCASTING);
An interactive streaming channel has two client roles: BROADCASTER
and AUDIENCE
, and the default role is AUDIENCE
. After setting the channel profile to LIVE_BROADCASTING
, your app may use the following steps to set the client role:
SetClientRole
method and pass in the client role set by the user.Note that in an interactive live streaming, only the host can be heard and seen. If you want to switch the client role after joining the channel, call the SetClientRole
method.
// Set the client role as the host.
mRtcEngine.SetClientRole(CLIENT_ROLE.BROADCASTER);
After setting the channel profile and client role, set the local video view before joining the channel so that the host can see the local video in the interactive streaming. Follow these steps to configure the local video:
Call EnableVideo
to enable the video module.
Call EnableVideoObserver
to get the local video.
// Enable the video module.
mRtcEngine.EnableVideo();
// Get the local video and pass it on to Unity.
mRtcEngine.EnableVideoObserver();`
Choose an object to display the local video, and drag the VideoSurface.cs file to Script, so that you can bind the VideoSurface.cs file to the object and see the local video. Unity renders 3D objects by default, such as Cube, Cylinder and Plane. To render the object in other types, modify the renderer in the VideoSurface.cs file.
// The default renderer is Renderer.
Renderer rend = GetComponent();
rend.material.mainTexture = nativeTexture;
// Change the renderer to RawImage.
RawImage rend = GetComponent();
rend.texture = nativeTexture;`
After setting the client role and the local video view (for the interactive video streaming), you can call JoinChannelByKey
to join a channel. Set the following parameters when calling this method:
channelKey
: The token for identifying the role and privileges of a user. Set it as one of the following values:
channelKey
as "".channelName
: The unique name of the channel to join. Users that input the same channel name join the same channel.
uid
: Integer. The unique ID of the local user. If you set uid
as 0
, the SDK automatically assigns one user ID and returns it in the OnJoinChannelSuccessHandler
callback.
If an interactive streaming channel uses both the RTC Native SDK and the RTC Web SDK, ensure that you call the EnableWebSdkInteroperability
method before joining the channel.
// Call this method to enable interoperability between the RTC Native SDK and the RTC Web SDK if the RTC Web SDK is in the channel.
mRtcEngine.EnableWebSdkInteroperability(true);
// Join a channel.
mRtcEngine.JoinChannelByKey(null, channel, null, 0);
In the interactive live video streaming you should also be able to see all the hosts, regardless of your role. This is achieved by calling SetForUser
in VideoSurface.cs after joining a channel.
Shortly after a remote host joins the channel, the SDK gets the host's user ID in the OnUserJoinedHandler
callback, and you can get the host’s uid
.
In the VideoSurface.cs file, call the SetForUser
method in the callback, and pass in the uid
to set the video of the remote user.
Drag the VideoSurface.cs script to the target object, so that you can bind the VideoSurface.cs to the object and see the remote video.
The video capture and render frame rate of Agora Unity SDK is 15 fps by default. Call SetGameFps
in the VideoSurface.cs file to adjust the video refresh rate based on your scenario.
// This callback occurs when the first video frame of a remote host is received and decoded after the remote host successfully joins the channel.
// You can call the SetForUser method in this callback to set up the remote video view.
private void OnUserJoinedHandler(uint uid, int elapsed)
{
Debug.Log ("OnUserJoinedHandler: uid = " + uid)
GameObject go = GameObject.Find (uid.ToString ());
if (!ReferenceEquals (go, null)) {
return;
}
go = GameObject.CreatePrimitive (PrimitiveType.Plane);
if (!ReferenceEquals (go, null)) {
go.name = uid.ToString ();
VideoSurface remoteVideoSurface = go.AddComponent<VideoSurface> ();
// Set the remote video.
remoteVideoSurface.SetForUser (uid);
// Adjust the video refreshing frame rate. The video capture and render frame rate of Agora Unity SDK is 15 fps by default.
// For example, in the game scenario, the refreshing frame rate of the game is 60 fps, which causes 3 times redundancy compared to the video render frame rate. Call the SetGameFps to adjust the refreshing frame rate of the game to 15 fps.
remoteVideoSurface.SetGameFps (60);
remoteVideoSurface.SetEnable (true);
}
mRemotePeer = uid;
}
To remove the VideoSurface.cs script from the object, see the following sample codes.
private void OnUserOfflineHandler(uint uid, USER_OFFLINE_REASON reason)
{
// Remove video stream.
// Call this in the main thread.
GameObject go = GameObject.Find (uid.ToString());
if (!ReferenceEquals (go, null)) {
Destroy (go);
}
}
According to your scenario, such as when the interactive live streaming ends and when you need to close the app, call LeaveChannel
to leave the current channel, and call DisableVideoObserver
to disable the video.
public void leave()
{
Debug.Log ("calling leave");
if (mRtcEngine == null)
return;
// Leave the channel.
mRtcEngine.LeaveChannel();
// Disable the video.
mRtcEngine.DisableVideoObserver();
}
After leaving the channel, if you want to exit the app or release the memory of IRtcEngine
, call Destroy
to destroy the IRtcEngine
object.
void OnApplicationQuit()
{
if (mRtcEngine != null)
{
// Destroy the IRtcEngine object.
IRtcEngine.Destroy();
mRtcEngine = null;
}
}
Run the project in Unity. When you set the role as the host and successfully start the interactive live video streaming, you can see the video view of yourself in the app. When you set the role as the audience and successfully join the interactive live video streaming, you can see the video view of the host in the app.
When using the Agora Unity SDK, you can also use the dollowing documents as a reference: