Stagefright框架中视频播放流程

Stagefright框架中视频播放流程

1.创建playerengine

// 设置数据源，以及 audio sink

MediaPlayer::SetDataSource(PATH_TO_FILE)->

MediaPlayerService::create->

MediaPlayerService::Client::setDataSource->

GetPlayerType->

MediaPlayerService:: Client::CreatePlayer->

StagefrightPlayer:: setAudioSink->

StagefrightPlayer:: setDataSource->

Create MediaPlayerImpl（AwesomePlayer）->

MediaPlayerImpl:: MediaPlayerImpl

PlayerType:

PV_PLAYER--------------------(OpenCore中的PVPlayer，2.2之前的默认多媒体框架，从2.3开始android源码中已经不存在了，变更为Stagefright，但芯片厂商会添加进来。位于external/opencore目录。)

SONIVOX_PLAYER----------- MidiFile()(MIDI 格式)

STAGEFRIGHT_PLAYER----- StagefrightPlayer

NU_PLAYER---------------------NuPlayer(流媒体播放器)

TEST_PLAYER------------------- TestPlayerStub (only for ‘test’ and ‘eng’build)

//以下为与openMax插件的初始化连接。

AwesomePlayer:mClient.connect()->

OMXClient::connect->

MediaPlayerService::getOMX()->

OMXMaster::OMXMaster: addVendorPlugin ()->

addPlugin((*createOMXPlugin ())->

*createOMXPlugin (){

new TIOMXPlugin;

}

2.解析mUri指定的内容，根据header来确定对应的Extractor

AwesomePlayer:: prepare()

AwesomePlayer:: prepareAsync_l()->

在该函数中启动mQueue,作为EventHandler(stagefright使用event来进行驱动)

AwesomePlayer::finishSetDataSource_l()->

MediaExtractor::create(datasource)->

3.使用extractor对文件做A/V分离(mVideoTrack/mAudioTrack)

AwesomePlayer::setDataSource_l(extractor)->

AwesomePlayer::setVideoSource()->

AwesomePlayer::setAudioSource()->

mVideoTrack=source

mAudioTrack=source

4.根据mVideoTrace中编码类型来选择video_decoder(mVideoSource)

AwesomePlayer::initVideoDecoder()->

mVideoSource->start();(初始化解码器)

OMXCodec::Create()->

根据编码类型去匹配codecs,将softwareCodec优先放在matchCodecs前面，优先匹配，即优先建立softWareCodec

<MediaSource>softwareCodec=InstantiateSoftwareCodec(componentName, source)->

如果没有匹配的softWareCodec则去调用hardware中实现的omx_codec

omx->allocateNode(componentName...)->

sp<OMXCodec> codec = new OMXCodec(~)->

observer->setCodec(codec)->

err = codec->configureCodec(meta, flags)->

return codec.

5.根据mAudioTrace中编码类型来选择audio_decoder(mAudioSource)

AwesomePlayer::initAudioDecoder()->

mAudioSource->start();(初始化解码器)

OMXCodec::Create()->

根据编码类型去匹配codecs,将softwareCodec优先放在matchCodecs前面，优先匹配，即优先建立softWareCodec

<MediaSource>softwareCodec=InstantiateSoftwareCodec(componentName, source)->

如果没有匹配的softWareCodec则去调用Hardware中实现的omx_codec

omx->allocateNode(componentName...)->

sp<OMXCodec> codec = new OMXCodec(~)->

observer->setCodec(codec)->

err = codec->configureCodec(meta, flags)->

return codec.

6.创建AudioPlayer,解码并开启Audio output播放audio数据

AwesomePlayer::play_l->

mAudioPlayer = new AudioPlayer(mAudioSink, this);

mAudioPlayer->setSource(mAudioSource);

mAudioPlayer->start

mSource->read(&mFirstBuffer);(在audioplayer启动过程中，会先读取第一段需解码后的资料。)

mAudioSink->open(..., &AudioPlayer::AudioSinkCallback, ...);

AudioSinkCallback{

me->fillBuffer(buffer, size)

}

开启audio output，同时AudioPlayer将callback函数设给它，之后每次callback函数被调用，AudioPlayer便会去读取Audio decoder解码后的资料。)

7.根据Codec类型选择Renderer

AwesomePlayer：:start->

postVideoEvent_l();

AwesomePlayer::onVideoEvent()->

mVideoSource->read()(&mVideoBuffer, &options)->

AwesomePlayer::initRenderer_l()->

判断Codec类型，

HardWare Codec：

mVideoRenderer =new AwesomeNativeWindowRenderer(mSurface, rotationDegrees);

AwesomeNativeWindowRenderer::render()(hook Called by EGL)->

HardWare Codec不需要进行ColorConvert操作，直接push到NativeWindow

SoftWare Codec：

mVideoRenderer = new AwesomeLocalRenderer(mSurface, meta)->

mVideoRenderer = new SoftwareRenderer()->

SoftwareRenderer::render()->

AwesomePlayer::onVideoEvent()->

[Check Timestamp]

mVideoRenderer->render(mVideoBuffer);

8.Audio和Video同步

Stagefright中Audio由CallBack驱动数据流，Video则在OnVideoEvent中获取Audio的timeStamp，进行同步。

Audio::fillBuffer()->

mPositionTimeMediaUs为资料中的timestamp,

mPositionTimeRealUs为播放资料的实际时间。

AwesomePlayer::onVideoEvent()->

mTimeSourceDeltaUs = realTimeUs- mediaTimeUs