During a video call or interactive streaming, sharing the screen enhances communication by displaying the speaker's screen on the display of other speakers or audience members in the channel.
Screen sharing is applied in the following scenarios:
The Agora Unity SDK provides two ways of screen sharing, namely app screen sharing and desktop screen sharing. Agora also provides open-source sample projects on GitHub that implement screen sharing on Unity in the two ways:
The following screen shots give you a glimpse of the two sample projects when implementing screen sharing.
Sharer | Viewer | |
---|---|---|
App | ||
Desktop |
Before sharing the screen, ensure that you have implemented the basic real-time communication functions in your project. See Start a Video Call or Start Interactive Video Streaming for details.
The differences of the two approaches for screen sharing are as follows:
Supported platforms | Application | Underlining support | |
---|---|---|---|
App screen sharing | Android, iOS, macOS, Windows | Current viewable screen of the Unity App | External video source |
Desktop screen sharing | Windows, macOS | Entire desktop or app region/window running on the platform | API call |
Refer to the following steps to share the screen via external video source in your project:
Specify the external video source by calling SetExternalVideoSource
before JoinChannelByKey
.
mRtcEngine.SetExternalVideoSource(true, false);
Define the Texture2D, and use Texture2D to read screen pixels as an external video source.
mRect = new Rect(0, 0, Screen.width, Screen.height);
mTexture = new Texture2D((int)mRect.width, (int)mRect.height, TextureFormat.RGBA32, false);
mTexture.ReadPixels(mRect, 0, 0);
mTexture.Apply();
Call PushVideoFrame
to send the video source to the SDK and to implement screen sharing.
int a = rtc.PushVideoFrame(externalVideoFrame);
The following diagram shows the API call sequence for sharing the screen via external video source.
Refer to the following code to share the screen in your project.
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using agora_gaming_rtc;
using UnityEngine.UI;
using System.Globalization;
using System.Runtime.InteropServices;
using System;
public class ShareScreen : MonoBehaviour
{
Texture2D mTexture;
Rect mRect;
[SerializeField]
private string appId = "Your_AppID";
[SerializeField]
private string channelName = "agora";
public IRtcEngine mRtcEngine;
int i = 100;
void Start()
{
Debug.Log("ScreenShare Activated");
mRtcEngine = IRtcEngine.GetEngine(appId);
// Sets the output log level of the SDK.
mRtcEngine.SetLogFilter(LOG_FILTER.DEBUG | LOG_FILTER.INFO | LOG_FILTER.WARNING | LOG_FILTER.ERROR | LOG_FILTER.CRITICAL);
// Enables the video module.
mRtcEngine.EnableVideo();
// Enables the video observer.
mRtcEngine.EnableVideoObserver();
// Configures the external video source.
mRtcEngine.SetExternalVideoSource(true, false);
// Joins a channel.
mRtcEngine.JoinChannel(channelName, null, 0);
// Creates a rectangular region of the screen.
mRect = new Rect(0, 0, Screen.width, Screen.height);
// Creates a texture of the rectangle you create.
mTexture = new Texture2D((int)mRect.width, (int)mRect.height, TextureFormat.RGBA32, false);
}
void Update()
{
StartCoroutine(shareScreen());
}
// Starts to share the screen.
IEnumerator shareScreen()
{
yield return new WaitForEndOfFrame();
// Reads the pixels of the rectangle you create.
mTexture.ReadPixels(mRect, 0, 0);
// Applies the pixels read from the rectangle to the texture.
mTexture.Apply();
// Gets the raw texture data and apply it to an array of bytes.
byte[] bytes = mTexture.GetRawTextureData();
// Gives enough space for the bytes array.
int size = Marshal.SizeOf(bytes[0]) * bytes.Length;
// Checks whether the IRtcEngine instance exists.
IRtcEngine rtc = IRtcEngine.QueryEngine();
if (rtc != null)
{
// Creates a new external video frame.
ExternalVideoFrame externalVideoFrame = new ExternalVideoFrame();
// Sets the buffer type of the video frame.
externalVideoFrame.type = ExternalVideoFrame.VIDEO_BUFFER_TYPE.VIDEO_BUFFER_RAW_DATA;
// Sets the format of the video pixel.
externalVideoFrame.format = ExternalVideoFrame.VIDEO_PIXEL_FORMAT.VIDEO_PIXEL_RGBA;
// Applies the raw data.
externalVideoFrame.buffer = bytes;
// Sets the width (pixel) of the video frame.
externalVideoFrame.stride = (int)mRect.width;
// Sets the height (pixel) of the video frame.
externalVideoFrame.height = (int)mRect.height;
// Removes pixels from the sides of the frame
externalVideoFrame.cropLeft = 10;
externalVideoFrame.cropTop = 10;
externalVideoFrame.cropRight = 10;
externalVideoFrame.cropBottom = 10;
// Rotates the video frame (0, 90, 180, or 270)
externalVideoFrame.rotation = 180;
// Increments i with the video timestamp.
externalVideoFrame.timestamp = i++;
// Pushes the external video frame with the frame you create.
int a = rtc.PushVideoFrame(externalVideoFrame);
}
}
}
Agora provides the following APIs for you to implement screen share on the desktop:
You can also use the sample codes in DesktopScreenShare as a reference.