This page shows you how to get raw video data for pre- and post-processing.
During the video transmission process, you can pre- and post-process the captured video data to achieve the desired playback effect.
Agora provides the raw data function for you to process the video data as per your application scenario. This function enables you to pre-process the captured video frames before sending it to the encoder, or to post-process the decoded video frames.
Before using the raw data function, ensure that you have implemented the basic real-time communication functions in your project. See Start a Video Call or Start Interactive Live Video Streaming for details.
To implement the raw video data function in your project, refer to the following steps.
Before joining a channel, create an IVideoFrameObserver
object and then call registerVideoFrameObserver
to register video frame observer.
int ret = engine.registerVideoFrameObserver(iVideoFrameObserver);
Implement the onCaptureVideoFrame
, onRenderVideoFrame
, and onScreenCaptureVideoFrame
callbacks. Get the video frames captured the callbacks and process the data according to your needs.
private final IVideoFrameObserver iVideoFrameObserver = new IVideoFrameObserver() {
@Override
public boolean onCaptureVideoFrame(VideoFrame videoFrame) {
Log.i(TAG, "OnEncodedVideoImageReceived"+Thread.currentThread().getName());
if(isSnapshot){
isSnapshot = false;
// Get image bitmap
VideoFrame.Buffer buffer = videoFrame.getBuffer();
VideoFrame.I420Buffer i420Buffer = buffer.toI420();
int width = i420Buffer.getWidth();
int height = i420Buffer.getHeight();
ByteBuffer bufferY = i420Buffer.getDataY();
ByteBuffer bufferU = i420Buffer.getDataU();
ByteBuffer bufferV = i420Buffer.getDataV();
byte[] i420 = YUVUtils.toWrappedI420(bufferY, bufferU, bufferV, width, height);
Bitmap bitmap = YUVUtils.NV21ToBitmap(getContext(),
YUVUtils.I420ToNV21(i420, width, height),
width,
height);
Matrix matrix = new Matrix();
matrix.setRotate(270);
// Rotate around a specific spot
Bitmap newBitmap = Bitmap.createBitmap(bitmap, 0, 0, width, height, matrix, false);
// save to file
saveBitmap2Gallery(newBitmap);
bitmap.recycle();
// Release
i420Buffer.release();
}
return false;
}
@Override
public boolean onScreenCaptureVideoFrame(VideoFrame videoFrame) {
return false;
}
@Override
public boolean onMediaPlayerVideoFrame(VideoFrame videoFrame, int i) {
return false;
}
@Override
public boolean onRenderVideoFrame(String s, int i, VideoFrame videoFrame) {
return false;
}
@Override
public int getVideoFrameProcessMode() {
return 0;
}
@Override
public int getVideoFormatPreference() {
return 1;
}
@Override
public int getRotationApplied() {
return 0;
}
@Override
public boolean getMirrorApplied() {
return false;
}
};
This section includes in depth information about the methods you used in this page, and links to related pages.
Agora provides an open-source sample project Raw Video Data on GitHub.