做网站要找什么公司,wordpress 知更鸟5.2,广告平面设计师,快速排名软件案例系列文章目录
ExoPlayer架构详解与源码分析#xff08;1#xff09;——前言 ExoPlayer架构详解与源码分析#xff08;2#xff09;——Player ExoPlayer架构详解与源码分析#xff08;3#xff09;——Timeline ExoPlayer架构详解与源码分析#xff08;4#xff09;—…系列文章目录
ExoPlayer架构详解与源码分析1——前言 ExoPlayer架构详解与源码分析2——Player ExoPlayer架构详解与源码分析3——Timeline ExoPlayer架构详解与源码分析4——整体架构 ExoPlayer架构详解与源码分析5——MediaSource ExoPlayer架构详解与源码分析6——MediaPeriod ExoPlayer架构详解与源码分析7——SampleQueue ExoPlayer架构详解与源码分析8——Loader ExoPlayer架构详解与源码分析9——TsExtractor ExoPlayer架构详解与源码分析10——H264Reader ExoPlayer架构详解与源码分析11——DataSource ExoPlayer架构详解与源码分析12——Cache ExoPlayer架构详解与源码分析13——TeeDataSource和CacheDataSource ExoPlayer架构详解与源码分析14——ProgressiveMediaPeriod ExoPlayer架构详解与源码分析15——Renderer 文章目录 系列文章目录前言RendererBaseRendererMediaCodecMediaCodecRendererMediaCodecAudioRendererMediaCodecVideoRenderer参考时间戳的计算总结 前言
如果你已经看完理解了前面MediaSource的内容我相信你已经知道数据是如何获取并解析好放入到缓存了我们先跳过中间那些控制管理环节这些数据最终流入的方向就是本篇要讲的Renderer了。可以把Renderer想象成火箭的涡轮发动机从MediaSource那源源不断的获取燃料在发动机里点火燃烧为火箭升空提供强大的动力。和火箭一样要想升空发动机必须平稳再火箭运行的不通时间精确的执行预设好的动作。这就需要一个良好的时间同步设计而Renderer的核心内容就是同步。
Renderer
渲染从SampleStream读取的媒体。 在内部渲染器的生命周期由所属的ExoPlayer管理。随着整体播放状态和启用的轨道的变化渲染器会在各种状态之间转换。有效的状态转换如下所示并用每次转换期间调用的方法进行注释。 看下主要方法
init 初始化Renderer入参index为当前Renderer在所有Renderer中的索引入参playerId为当前播放器的IDenable 使渲染器能够使用传入的SampleStream当Renderer 处于Disabled状态是才可能被调用start 启动渲染器这意味着对render的调用将导致媒体被渲染。当Renderer 处于Enable状态时才能调用此方法render 增量渲染SampleStream 。当渲染器处于以下Enable、 Started状态时可以调用此方法 。replaceStream 替换SampleStream 。当渲染器处于以下Enable、 Started状态时可以调用此方法 。
Renderer对象创建完成一般先调init方法初始化然后调用enable传入SampleStreamenable内部会调用replaceStream初始化SampleStream之后调用start将状态置为Started最后调用render方法开始渲染
再看下Renderer模块的整体结构 Renderer直接由抽象类BaseRenderer实现下面的MediaCodecRenderer音视频、TextRenderer字幕、MetadataRendererMeta信息、CameraMotionRenderer镜头信息用于VR全景之类的数据对应各种类型轨道的渲染器。本文篇幅有限主要介绍音视频也就是MediaCodecRenderer其他的Renderer感兴趣的可以自行研究。可以看到MediaCodecRenderer下又分视频Video和音频Audio两大块视频最终交给Android系统的MediaCodec来处理而音频最终交由Android系统的AudioTrack处理。
BaseRenderer
Renderer的直接实现类主要用于一些状态的控制存储和一些全局变量的管理 Overridepublic final void init(int index, PlayerId playerId) {this.index index;this.playerId playerId;}Overridepublic final void enable(RendererConfiguration configuration,//renderer配置信息Format[] formats,//轨道信息SampleStream stream,//待渲染数据long positionUs,//当前播放位置boolean joining,//是否启用此渲染器来加入正在进行的播放boolean mayRenderStartOfStream,//即使状态尚未STATE_STARTED 是否允许此渲染器渲染流的开头。long startPositionUs,//渲染的开始位置long offsetUs)//在渲染之前添加到从stream读取的缓冲区时间戳的偏移量。throws ExoPlaybackException {Assertions.checkState(state STATE_DISABLED);this.configuration configuration;state STATE_ENABLED;onEnabled(joining, mayRenderStartOfStream);//调用子类replaceStream(formats, stream, startPositionUs, offsetUs);resetPosition(positionUs, joining);}Overridepublic final void replaceStream(Format[] formats, SampleStream stream, long startPositionUs, long offsetUs)throws ExoPlaybackException {Assertions.checkState(!streamIsFinal);this.stream stream;//替换当前的全局SampleSteamif (readingPositionUs C.TIME_END_OF_SOURCE) {readingPositionUs startPositionUs;}streamFormats formats;streamOffsetUs offsetUs;onStreamChanged(formats, startPositionUs, offsetUs);//子类实现}Overridepublic final void start() throws ExoPlaybackException {Assertions.checkState(state STATE_ENABLED);state STATE_STARTED;//改变状态onStarted();//子类实现}//BaseRenderer还提供了readSource方法用于读取Sample中的数据//readFlags知道当前需要获取的数据类型protected final ReadDataResult int readSource(FormatHolder formatHolder, DecoderInputBuffer buffer, ReadFlags int readFlags) {ReadDataResult//这里的stream最终从上文讲的SampleQueue中获取数据int result Assertions.checkNotNull(stream).readData(formatHolder, buffer, readFlags);if (result C.RESULT_BUFFER_READ) {//当前获取的是BUFFER数据if (buffer.isEndOfStream()) {readingPositionUs C.TIME_END_OF_SOURCE;return streamIsFinal ? C.RESULT_BUFFER_READ : C.RESULT_NOTHING_READ;}buffer.timeUs streamOffsetUs;readingPositionUs max(readingPositionUs, buffer.timeUs);} else if (result C.RESULT_FORMAT_READ) {//当前获取的是Format数据Format format Assertions.checkNotNull(formatHolder.format);if (format.subsampleOffsetUs ! Format.OFFSET_SAMPLE_RELATIVE) {format format.buildUpon().setSubsampleOffsetUs(format.subsampleOffsetUs streamOffsetUs).build();formatHolder.format format;}}return result;}BaseRenderer的实现比较简单重点看下它的子类这里主要学习下MediaCodecRenderer在看MediaCodecRenderer前得先了解下Android系统的MediaCodec。
MediaCodec
MediaCodec 可用于访问低级媒体编解码器即编码器/解码器组件。它是 Android 低级多媒体支持基础设施的一部分这里主要了解下解码过程不知道还有没有读者记得这张在讲SampleQueue里出现的图了 解码器的作用就是处理输入的编码数据输出解码后的数据。它使用一组输入和输出缓冲区异步处理数据。首先调用者初始化MediaCodec.configure向MediaCodec.dequeueInputBuffer请求一个空的输入缓冲区将其他地方读取到的编码数据填充输入缓冲区queueInputBuffer将其发送给MediaCodec进行处理。MediaCodec获取的输入缓冲数据进行解码完成后将解码后的数据输出至输出缓冲区。最后调用者向MediaCodec.dequeueOutputBuffer请求已填充的输出缓冲区调用者将获取输出缓冲区的解码数据将其渲染到指定地方使用完成后releaseOutputBuffer将其释放回MediaCodec。如果在MediaCodec.configure传入了SurfacereleaseOutputBuffer后会将解码数据直接渲染到传入的Surface上。
在MediaCodec生命周期中存在以下三种状态Stopped、Executing 、 Released。 Stopped 状态实际上是三个状态的组合Uninitialized、Configured 和 Error而 Executing 状态会经历三个子状态Flushed、Running 和 End-of-Stream。 当创建MediaCodec时编解码器处于Uninitialized状态。首先您需要通过configure对其进行配置这会将其置于Configured 状态然后调用start将其置于Executing 状态。在Executing 状态下就可以通过上述缓冲区队列操作来处理数据了。 Executing 状态具有三个子状态Flushed、Running 和 End-of-Stream。在 start之后MediaCodec立即处于 Flushed 子状态其中保存所有缓冲区。一旦第一个输入缓冲区出队编解码器就会进入Running 子状态大部分时间会执行在此状态下。当使用BUFFER_FLAG_END_OF_STREAM Flag标记进行MediaCodec.queueInputBuffer时MediaCodec将转换到End-of-Stream子状态。在此状态下编解码器不再接受更多输入缓冲区但仍生成输出缓冲区直到输出到达流末尾。对于解码器可以在处于 Executing 状态时随时使用 flash返回到 Flushed 子状态。 调用 stop 将编解码器返回到Uninitialized状态然后可以再次configure它。使用完编解码器后必须通过调用release来释放它。 有了上面的知识来看看看MediaCodecRenderer是如何使用这些方法完成整个渲染的。
MediaCodecRenderer
MediaCodecRenderer主要是通过Android的MediaCodec来渲染解码渲染出音视频内容主要有2个子类MediaCodecVideoRenderer和MediaCodecAudioRenderer。直接看下render的实现 Override//positionUs为当前的播放时间戳如果有音轨会获取音轨的PTS//elapsedRealtimeUs循环调用render开始前的时间戳组要用来计算程序的执行时长public void render(long positionUs, long elapsedRealtimeUs) throws ExoPlaybackException {...// We have a format.maybeInitCodecOrBypass();//如果是直出的也就是不需要Codec解码的数据if (bypassEnabled) {TraceUtil.beginSection(bypassRender);//while (bypassRender(positionUs, elapsedRealtimeUs)) {}TraceUtil.endSection();} else if (codec ! null) {//需要通过MeidaCodec解码的数据//记录循环开始时间用于计算执行时间是否超过renderTimeLimitMs决定是否继续循环long renderStartTimeMs SystemClock.elapsedRealtime();TraceUtil.beginSection(drainAndFeed);//先从MediaCodec中获取已解码数据while (drainOutputBuffer(positionUs, elapsedRealtimeUs) shouldContinueRendering(renderStartTimeMs)) {}//向MediaCodec输入待解码数据while (feedInputBuffer() shouldContinueRendering(renderStartTimeMs)) {}TraceUtil.endSection();}...}protected final void maybeInitCodecOrBypass() throws ExoPlaybackException {...if (isBypassPossible(inputFormat)) {initBypass(inputFormat);//对于不需要Codec解码的数据直接初始化Bypass主要就是初始化Byapass的bufferbypassBatchBufferreturn;}...//初始化MediaCodecmaybeInitCodecWithFallback(mediaCrypto, mediaCryptoRequiresSecureDecoder);...}private void maybeInitCodecWithFallback(Nullable MediaCrypto crypto, boolean mediaCryptoRequiresSecureDecoder)throws DecoderInitializationException {if (availableCodecInfos null) {try {//通过输入的数据的Meta信息获取用于初始化Codec的相关数据ListMediaCodecInfo allAvailableCodecInfos getAvailableCodecInfos(mediaCryptoRequiresSecureDecoder);...}...//开始初始化CodecinitCodec(codecInfo, crypto);
...}private void initCodec(MediaCodecInfo codecInfo, Nullable MediaCrypto crypto) throws Exception {//通过子类获取MediaCodecAdapter.ConfigurationMediaCodecAdapter.Configuration configuration getMediaCodecConfiguration(codecInfo, inputFormat, crypto, codecOperatingRate);...//创建出MediaCodecAdapter这里的Adapter主要有2个实现一个是SynchronousMediaCodecAdapter 通过同步的方式调用MediaCodec一个是针对API23的异步MeidaCodec调用的AsynchronousMediaCodecAdaptertry {TraceUtil.beginSection(createCodec: codecName);codec codecAdapterFactory.createAdapter(configuration);} finally {TraceUtil.endSection();}...}//这里为了方便看下SynchronousMediaCodecAdapter public MediaCodecAdapter createAdapter(Configuration configuration) throws IOException {Nullable MediaCodec codec null;try {//主要通过MediaCodec.createByCodecName(codecName)创建出MediaCodeccodec createCodec(configuration);TraceUtil.beginSection(configureCodec);//配置MediaCodeccodec.configure(//格式其中KEY_MAX_INPUT_SIZE确定了缓冲区的大小对应于format.maxInputSize可以查看计算逻辑configuration.mediaFormat,configuration.surface,//渲染的surfaceconfiguration.crypto,configuration.flags);TraceUtil.endSection();TraceUtil.beginSection(startCodec);//到这里MediaCodec就已经准备好了随时可以用来解码了codec.start();TraceUtil.endSection();return new SynchronousMediaCodecAdapter(codec);} catch (IOException | RuntimeException e) {if (codec ! null) {codec.release();}throw e;}}//直出数据渲染private boolean bypassRender(long positionUs, long elapsedRealtimeUs)throws ExoPlaybackException {...//有数据后调用子类processOutputBuffer渲染数据if (bypassBatchBuffer.hasSamples()) {if (processOutputBuffer(//这里调用MediaCodecAudioRendererpositionUs,elapsedRealtimeUs,/* codec */ null,bypassBatchBuffer.data,outputIndex,/* bufferFlags */ 0,bypassBatchBuffer.getSampleCount(),bypassBatchBuffer.getFirstSampleTimeUs(),bypassBatchBuffer.isDecodeOnly(),bypassBatchBuffer.isEndOfStream(),outputFormat)) {// The batch buffer has been fully processed.onProcessedOutputBuffer(bypassBatchBuffer.getLastSampleTimeUs());bypassBatchBuffer.clear();} else {// Could not process the whole batch buffer. Try again later.return false;}}...// 第一次先读取Sample数据到bypassBatchBufferbypassRead();
...}private boolean drainOutputBuffer(long positionUs, long elapsedRealtimeUs)throws ExoPlaybackException {//如果成功渲染了OutputBufferOutputBuffer会重置这里会hasOutputBufferfalse会拉取下一段Buffer继续执行//如果当前的Buffer因某种原因如渲染过快需要等待这个时候OutputBuffer还是上次未渲染的数据if (!hasOutputBuffer()) {int outputIndex;...//获取解码后的OutputBuffer的索引outputIndex codec.dequeueOutputBufferIndex(outputBufferInfo);}if (outputIndex 0) {//异常情况//格式变更if (outputIndex MediaCodec.INFO_OUTPUT_FORMAT_CHANGED /* (-2) */) {processOutputMediaFormatChanged();return true;}// MediaCodec.INFO_TRY_AGAIN_LATER (-1) or unknown negative return value.if (codecNeedsEosPropagation (inputStreamEnded || codecDrainState DRAIN_STATE_WAIT_END_OF_STREAM)) {processEndOfStream();}return false;}// 跳过特殊机型的适配数据if (shouldSkipAdaptationWorkaroundOutputBuffer) {shouldSkipAdaptationWorkaroundOutputBuffer false;codec.releaseOutputBuffer(outputIndex, false);return true;} else if (outputBufferInfo.size 0 (outputBufferInfo.flags MediaCodec.BUFFER_FLAG_END_OF_STREAM) ! 0) {// dequeued buffer 标记了结束Flag立即结束processEndOfStream();return false;}this.outputIndex outputIndex;//更新全局的outputIndex//通过outputIndex 获取OutputBufferoutputBuffer codec.getOutputBuffer(outputIndex);//根据outputBufferInfo初始化outputBufferif (outputBuffer ! null) {outputBuffer.position(outputBufferInfo.offset);outputBuffer.limit(outputBufferInfo.offset outputBufferInfo.size);}...}boolean processedOutputBuffer;if (codecNeedsEosOutputExceptionWorkaround codecReceivedEos) {processedOutputBuffer processOutputBuffer(//调用子类处理OutputBufferpositionUs,elapsedRealtimeUs,codec,outputBuffer,outputIndex,outputBufferInfo.flags,/* sampleCount */ 1,outputBufferInfo.presentationTimeUs,isDecodeOnlyOutputBuffer,isLastOutputBuffer,outputFormat);}//成功渲染了当前OutputBufferif (processedOutputBuffer) {onProcessedOutputBuffer(outputBufferInfo.presentationTimeUs);boolean isEndOfStream (outputBufferInfo.flags MediaCodec.BUFFER_FLAG_END_OF_STREAM) ! 0;resetOutputBuffer();//重置OutputBufferif (!isEndOfStream) {return true;}processEndOfStream();}//否则返回false中止drainOutputBuffer进入feedInputBufferreturn false;}//向MediaCodec输入数据private boolean feedInputBuffer() throws ExoPlaybackException {...if (inputIndex 0) {//如果已经resetInputBuffer//获取InputBuffer索引inputIndex codec.dequeueInputBufferIndex();//这里可能获取不到有可能MediaCodec的缓存已经满了此时就不再读取数据输入了//这个MediaCodec最大的缓存大小是在MediaCodec初始化时传入Format时确定的if (inputIndex 0) {return false;}//通过索引获取InputBufferbuffer.data codec.getInputBuffer(inputIndex);//清空数据buffer.clear();}//需要消耗当前InputBufferif (codecDrainState DRAIN_STATE_SIGNAL_END_OF_STREAM) {// We need to re-initialize the codec. Send an end of stream signal to the existing codec so// that it outputs any remaining buffers before we release it.if (codecNeedsEosPropagation) {// Do nothing.} else {codecReceivedEos true;codec.queueInputBuffer(inputIndex, 0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);resetInputBuffer();}codecDrainState DRAIN_STATE_WAIT_END_OF_STREAM;return false;}//特殊适配在InputBuffer前加入三个H.264 NAL单元SPS、PPS和 32 * 32 像素 IDR slice可以强制Format更新if (codecNeedsAdaptationWorkaroundBuffer) {codecNeedsAdaptationWorkaroundBuffer false;buffer.data.put(ADAPTATION_WORKAROUND_BUFFER);codec.queueInputBuffer(inputIndex, 0, ADAPTATION_WORKAROUND_BUFFER.length, 0, 0);resetInputBuffer();codecReceivedBuffers true;return true;}//对于自适应重配置解码器期望在缓冲区的开头提供重配置数据if (codecReconfigurationState RECONFIGURATION_STATE_WRITE_PENDING) {for (int i 0; i codecInputFormat.initializationData.size(); i) {byte[] data codecInputFormat.initializationData.get(i);buffer.data.put(data);}codecReconfigurationState RECONFIGURATION_STATE_QUEUE_PENDING;}int adaptiveReconfigurationBytes buffer.data.position();FormatHolder formatHolder getFormatHolder();SampleStream.ReadDataResult int result;try {//开始读取数据到InputBuffer里result readSource(formatHolder, buffer, /* readFlags */ 0);} catch (InsufficientCapacityException e) {onCodecError(e);//对于过大的Sample直接读取Mate信息跳过数据Sample读取readSourceOmittingSampleData(/* readFlags */ 0);flushCodec();return true;}
...if (result C.RESULT_FORMAT_READ) {//读取的是Mate信息if (codecReconfigurationState RECONFIGURATION_STATE_QUEUE_PENDING) {// We received two formats in a row. Clear the current buffer of any reconfiguration data// associated with the first format.buffer.clear();codecReconfigurationState RECONFIGURATION_STATE_WRITE_PENDING;}onInputFormatChanged(formatHolder);return true;}...long presentationTimeUs buffer.timeUs;//获取PTS...largestQueuedPresentationTimeUs max(largestQueuedPresentationTimeUs, presentationTimeUs);buffer.flip();//切换为readif (buffer.hasSupplementalData()) {handleInputBufferSupplementalData(buffer);}onQueueInputBuffer(buffer);try {//将包含未解码数据的InputBuffer传给MediaCodec解码if (bufferEncrypted) {codec.queueSecureInputBuffer(inputIndex, /* offset */ 0, buffer.cryptoInfo, presentationTimeUs, /* flags */ 0);} else {codec.queueInputBuffer(inputIndex, /* offset */ 0, buffer.data.limit(), presentationTimeUs, /* flags */ 0);}} catch (CryptoException e) {throw createRendererException(e, inputFormat, Util.getErrorCodeForMediaDrmErrorCode(e.getErrorCode()));}resetInputBuffer();//重置InputBuffer下次会读取新的InputBuffercodecReceivedBuffers true;codecReconfigurationState RECONFIGURATION_STATE_NONE;decoderCounters.queuedInputBufferCount;return true;}processOutputBuffer交由子类实现也就是MediaCodecVideoRenderer和MediaCodecAudioRendererMediaCodecAudioRenderer实现相对MediaCodecVideoRenderer简单
MediaCodecAudioRenderer
音频渲染器主要通过Android系统的AudioTrack实现音频播放 Overrideprotected boolean processOutputBuffer(long positionUs,long elapsedRealtimeUs,Nullable MediaCodecAdapter codec,Nullable ByteBuffer buffer,int bufferIndex,int bufferFlags,int sampleCount,long bufferPresentationTimeUs,boolean isDecodeOnlyBuffer,boolean isLastBuffer,Format format)throws ExoPlaybackException {checkNotNull(buffer);if (decryptOnlyCodecFormat ! null//包含编解码器初始化/编解码器特定数据而不是媒体数据直接release掉 (bufferFlags MediaCodec.BUFFER_FLAG_CODEC_CONFIG) ! 0) {// Discard output buffers from the passthrough (raw) decoder containing codec specific data.checkNotNull(codec).releaseOutputBuffer(bufferIndex, false);return true;}if (isDecodeOnlyBuffer) {//无需渲染的数据直接release掉if (codec ! null) {codec.releaseOutputBuffer(bufferIndex, false);}decoderCounters.skippedOutputBufferCount sampleCount;audioSink.handleDiscontinuity();return true;}boolean fullyConsumed;try {//开始渲染出数据调用DefaultAudioSink播放这些数据fullyConsumed audioSink.handleBuffer(buffer, bufferPresentationTimeUs, sampleCount);} catch (InitializationException e) {throw createRendererException(e, inputFormat, e.isRecoverable, PlaybackException.ERROR_CODE_AUDIO_TRACK_INIT_FAILED);} catch (WriteException e) {throw createRendererException(e, format, e.isRecoverable, PlaybackException.ERROR_CODE_AUDIO_TRACK_WRITE_FAILED);}if (fullyConsumed) {//渲染完毕releaseif (codec ! null) {codec.releaseOutputBuffer(bufferIndex, false);}decoderCounters.renderedOutputBufferCount sampleCount;return true;}return false;}//DefaultAudioSinkOverrideSuppressWarnings(ReferenceEquality)public boolean handleBuffer(ByteBuffer buffer, long presentationTimeUs, int encodedAccessUnitCount)throws InitializationException, WriteException {...//初始化创建出AudioTrack对象if (!isAudioTrackInitialized()) {try {if (!initializeAudioTrack()) {// Not yet ready for initialization of a new AudioTrack.return false;}} catch (InitializationException e) {if (e.isRecoverable) {throw e; // Do not delay the exception if it can be recovered at higher level.}initializationExceptionPendingExceptionHolder.throwExceptionIfDeadlineIsReached(e);return false;}}initializationExceptionPendingExceptionHolder.clear();if (startMediaTimeUsNeedsInit) {//首次执行startMediaTimeUs max(0, presentationTimeUs);startMediaTimeUsNeedsSync false;startMediaTimeUsNeedsInit false;if (useAudioTrackPlaybackParams()) {setAudioTrackPlaybackParametersV23();}applyAudioProcessorPlaybackParametersAndSkipSilence(presentationTimeUs);//audioTrack.play()if (playing) {play();}}
...// 校验 presentationTimeUslong expectedPresentationTimeUs startMediaTimeUs//播放开始时间 configuration.inputFramesToDurationUs(//帧数除以音频采样率计算出到这一帧的标准时长getSubmittedFrames() - trimmingAudioProcessor.getTrimmedFrameCount());if (!startMediaTimeUsNeedsSync//和计算的标准时间相差了200ms Math.abs(expectedPresentationTimeUs - presentationTimeUs) 200000) {if (listener ! null) {listener.onAudioSinkError(new AudioSink.UnexpectedDiscontinuityException(presentationTimeUs, expectedPresentationTimeUs));}startMediaTimeUsNeedsSync true;//标记开始时间需要同步}if (startMediaTimeUsNeedsSync) {//同步startMediaTimeUsif (!drainToEndOfStream()) {// Dont update timing until pending AudioProcessor buffers are completely drained.return false;}// 开始调整startMediaTimeUs//获取时间差long adjustmentUs presentationTimeUs - expectedPresentationTimeUs;//重新设置startMediaTimeUsstartMediaTimeUs adjustmentUs;startMediaTimeUsNeedsSync false;// Re-apply playback parameters because the startMediaTimeUs changed.applyAudioProcessorPlaybackParametersAndSkipSilence(presentationTimeUs);if (listener ! null adjustmentUs ! 0) {listener.onPositionDiscontinuity();}}//总提交帧数增加if (configuration.outputMode OUTPUT_MODE_PCM) {submittedPcmBytes buffer.remaining();} else {submittedEncodedFrames (long) framesPerEncodedSample * encodedAccessUnitCount;}inputBuffer buffer;inputBufferAccessUnitCount encodedAccessUnitCount;}//最终调用audioTrack.write写入数据完成音频输出processBuffers(presentationTimeUs);if (!inputBuffer.hasRemaining()) {inputBuffer null;inputBufferAccessUnitCount 0;return true;}if (audioTrackPositionTracker.isStalled(getWrittenFrames())) {Log.w(TAG, Resetting stalled audio track);flush();return true;}return false;}可以看到MediaCodecAudioRenderer直接使用了输入的bufferPresentationTimeUs作为PTS将音频输出期间没有进行过调整只是调整了startMediaTimeUs 开始时间所以实现简单几乎不涉及任何的时间同步代码。这里可以确定Exoplayer可能采用音频的PTS作为参考时钟在播放视频时以音频时钟为准将视频时间同步到音频上。下面就证实下上面的猜想看下MediaCodecVideoRenderer的实现。
MediaCodecVideoRenderer
视频数据在调用MediaCodec.releaseOutputBuffer后就会渲染到指定的Surface上这个过程就主要执行在MediaCodecVideoRenderer里
Overrideprotected boolean processOutputBuffer(long positionUs,//参考时钟的播放位置对应Audio的PTSlong elapsedRealtimeUs,//循环调用render开始前的时间戳主要用来计算程序的执行时长Nullable MediaCodecAdapter codec,Nullable ByteBuffer buffer,int bufferIndex,int bufferFlags,int sampleCount,long bufferPresentationTimeUs,//视频流的PTSboolean isDecodeOnlyBuffer,boolean isLastBuffer,Format format)throws ExoPlaybackException {checkNotNull(codec); // 视频必须要codec解码if (initialPositionUs C.TIME_UNSET) {initialPositionUs positionUs;//第一次初始化位置赋值}//更新上一次的bufferPresentationTimeUsif (bufferPresentationTimeUs ! lastBufferPresentationTimeUs) {if (!videoFrameProcessorManager.isEnabled()) {frameReleaseHelper.onNextFrame(bufferPresentationTimeUs);} // else, update the frameReleaseHelper when releasing the processed frames.this.lastBufferPresentationTimeUs bufferPresentationTimeUs;}//获取流开始PTSlong outputStreamOffsetUs getOutputStreamOffsetUs();//当前的PTS-开始PTSPTS时长long presentationTimeUs bufferPresentationTimeUs - outputStreamOffsetUs;if (isDecodeOnlyBuffer !isLastBuffer) {skipOutputBuffer(codec, bufferIndex, presentationTimeUs);return true;}// Note: Use of double rather than float is intentional for accuracy in the calculations below.boolean isStarted getState() STATE_STARTED;//获取当前系统时间long elapsedRealtimeNowUs SystemClock.elapsedRealtime() * 1000;long earlyUs //提前时长使用当前流的PTS-参考时钟-程序执行时长calculateEarlyTimeUs(positionUs,elapsedRealtimeUs,elapsedRealtimeNowUs,bufferPresentationTimeUs,isStarted);if (displaySurface placeholderSurface) {// Skip frames in sync with playback, so well be at the right frame if the mode changes.if (isBufferLate(earlyUs)) {skipOutputBuffer(codec, bufferIndex, presentationTimeUs);updateVideoFrameProcessingOffsetCounters(earlyUs);return true;}return false;}//当前帧已经延迟超过30ms(earlyUs-30000),且距离上次渲染时间已经超过了100ms此时画面是静止的需要强制去渲染当前帧boolean forceRenderOutputBuffer shouldForceRender(positionUs, earlyUs);if (forceRenderOutputBuffer) {//强制渲染场景boolean notifyFrameMetaDataListener;if (videoFrameProcessorManager.isEnabled()) {notifyFrameMetaDataListener false;if (!videoFrameProcessorManager.maybeRegisterFrame(format, presentationTimeUs, isLastBuffer)) {return false;}} else {notifyFrameMetaDataListener true;}renderOutputBufferNow(//开始渲染codec, format, bufferIndex, presentationTimeUs, notifyFrameMetaDataListener);updateVideoFrameProcessingOffsetCounters(earlyUs);return true;}if (!isStarted || positionUs initialPositionUs) {return false;}// 计算提交给Codec也就是releaseOutputBuffer时指定的送显时间戳long systemTimeNs System.nanoTime();//当前时间提前的时长long unadjustedFrameReleaseTimeNs systemTimeNs (earlyUs * 1000);// 进一步调整精确送显时间戳后面会看到具体代码long adjustedReleaseTimeNs frameReleaseHelper.adjustReleaseTime(unadjustedFrameReleaseTimeNs);if (!videoFrameProcessorManager.isEnabled()) {//使用精确的送显时间重新计算earlyUsearlyUs (adjustedReleaseTimeNs - systemTimeNs) / 1000;}//丢帧逻辑boolean treatDroppedBuffersAsSkipped joiningDeadlineMs ! C.TIME_UNSET;if (shouldDropBuffersToKeyframe(earlyUs, elapsedRealtimeUs, isLastBuffer) maybeDropBuffersToKeyframe(positionUs, treatDroppedBuffersAsSkipped)) {return false;} else if (shouldDropOutputBuffer(earlyUs, elapsedRealtimeUs, isLastBuffer)) {if (treatDroppedBuffersAsSkipped) {skipOutputBuffer(codec, bufferIndex, presentationTimeUs);} else {dropOutputBuffer(codec, bufferIndex, presentationTimeUs);}updateVideoFrameProcessingOffsetCounters(earlyUs);return true;}...if (Util.SDK_INT 21) {// 大于等于21这里直接传入送显时间让Codec决定什么时候送显if (earlyUs 50000) {//舍弃提前太多的帧最多渲染提前50ms送显的帧if (adjustedReleaseTimeNs lastFrameReleaseTimeNs) {//2次送显时间一致说明渲染速率要比显示器刷新率快尽快跳过当前帧保证渲染速率skipOutputBuffer(codec, bufferIndex, presentationTimeUs);} else {//触发送显的监听notifyFrameMetadataListener(presentationTimeUs, adjustedReleaseTimeNs, format);//使用adjustedReleaseTimeNs送显时间releaseOutputBufferrenderOutputBufferV21(codec, bufferIndex, presentationTimeUs, adjustedReleaseTimeNs);}updateVideoFrameProcessingOffsetCounters(earlyUs);lastFrameReleaseTimeNs adjustedReleaseTimeNs;return true;}} else {// 21以下系统需要自己控制送显时间releaseOutputBuffer不支持传入送显时间if (earlyUs 30000) {//舍弃提前太多的帧最多渲染提前30ms送显的帧至于为啥是30ms问就是感觉if (earlyUs 11000) {//如果在11m到30ms之间还是有点早需要阻塞等待// Note: The 11ms threshold was chosen fairly arbitrarily.//11ms没有太多依据凭感觉try {// 保证至少1msThread.sleep((earlyUs - 10000) / 1000);} catch (InterruptedException e) {Thread.currentThread().interrupt();return false;}}//触发送显的监听notifyFrameMetadataListener(presentationTimeUs, adjustedReleaseTimeNs, format);//低于11m的就直接送显了renderOutputBuffer(codec, bufferIndex, presentationTimeUs);//直接送显updateVideoFrameProcessingOffsetCounters(earlyUs);return true;}}// 返回false可能当前还未播放或者还没到渲染这帧的时间return false;}//计算提前时长private long calculateEarlyTimeUs(long positionUs,long elapsedRealtimeUs,long elapsedRealtimeNowUs,long bufferPresentationTimeUs,boolean isStarted) {// Note: Use of double rather than float is intentional for accuracy in the calculations below.double playbackSpeed getPlaybackSpeed();//计算比当前播放的时间提前了多久换句话说就是当前帧在真实需要渲染时间前提前了多久开始渲染。负值说明已经画面延迟了我们在需要的时间并没有提供相应的渲染数据。long earlyUs (long) ((bufferPresentationTimeUs - positionUs) / playbackSpeed);if (isStarted) {// 这里计算减去程序执行到这里所用的耗时earlyUs - elapsedRealtimeNowUs - elapsedRealtimeUs;}return earlyUs;}RequiresApi(21)protected void renderOutputBufferV21(MediaCodecAdapter codec, int index, long presentationTimeUs, long releaseTimeNs) {TraceUtil.beginSection(releaseOutputBuffer);codec.releaseOutputBuffer(index, releaseTimeNs);//这里传入了送显时间由底层控制送显时间TraceUtil.endSection();decoderCounters.renderedOutputBufferCount;consecutiveDroppedFrameCount 0;if (!videoFrameProcessorManager.isEnabled()) {lastRenderRealtimeUs SystemClock.elapsedRealtime() * 1000;maybeNotifyVideoSizeChanged(decodedVideoSize);maybeNotifyRenderedFirstFrame();}}protected void renderOutputBuffer(MediaCodecAdapter codec, int index, long presentationTimeUs) {TraceUtil.beginSection(releaseOutputBuffer);codec.releaseOutputBuffer(index, true);//低于21的系统这里直接就送显了true表示会渲染到Codec指定的Surface上TraceUtil.endSection();decoderCounters.renderedOutputBufferCount;consecutiveDroppedFrameCount 0;if (!videoFrameProcessorManager.isEnabled()) {lastRenderRealtimeUs SystemClock.elapsedRealtime() * 1000;maybeNotifyVideoSizeChanged(decodedVideoSize);maybeNotifyRenderedFirstFrame();}}
//看下VideoFrameReleaseHelper的进一步调整精确送显时间戳的过程public long adjustReleaseTime(long releaseTimeNs) {// Until we know better, the adjustment will be a no-op.long adjustedReleaseTimeNs releaseTimeNs;//同步状态下执行所谓Synced指获取到连续的15个帧间隔时间小于1ms的帧if (lastAdjustedFrameIndex ! C.INDEX_UNSET frameRateEstimator.isSynced()) {//用这些帧的总时常/帧数平局的帧间隔时长long frameDurationNs frameRateEstimator.getFrameDurationNs();long candidateAdjustedReleaseTimeNs lastAdjustedReleaseTimeNs//预测当前帧送显时间上次帧的送显时间当前帧到上次帧的帧数*帧间间隔时长/播放速度 (long) ((frameDurationNs * (frameIndex - lastAdjustedFrameIndex)) / playbackSpeed);//如果当前送显时间和预测的送显时间相隔时长小等于20ms则使用预测的送显时间//这里20ms主要是考虑Android VSYNC机制送显的数据不是立即显示到屏幕上而是经过3级的缓存在接收到VSYNC信号时才会显示到屏幕上也就是你期望的送显时间并不是实际的送显时间if (adjustmentAllowed(releaseTimeNs, candidateAdjustedReleaseTimeNs)) {adjustedReleaseTimeNs candidateAdjustedReleaseTimeNs;} else {resetAdjustment();}}pendingLastAdjustedFrameIndex frameIndex;pendingLastAdjustedReleaseTimeNs adjustedReleaseTimeNs;//下面是基于Vsync信号时间戳来调整送显时间戳保证帧数据尽快显示到屏幕上if (vsyncSampler null || vsyncDurationNs C.TIME_UNSET) {return adjustedReleaseTimeNs;}//获取当前的Vsync信号时间戳long sampledVsyncTimeNs vsyncSampler.sampledVsyncTimeNs;if (sampledVsyncTimeNs C.TIME_UNSET) {return adjustedReleaseTimeNs;}// 寻找距离当前送显时间戳最近的目标Vsync信号时间戳long snappedTimeNs closestVsync(adjustedReleaseTimeNs, sampledVsyncTimeNs, vsyncDurationNs);// 减去一个vsyncOffsetNs保证送显时间在前一个Vsync信号时间戳前目标Vsync信号时间戳之后//这个vsyncOffsetNs计算方式//1.获取当前屏幕的刷新率如60Hz就是屏幕每秒刷新60帧//2.计算每帧间隔时长60Hz每帧间隔就是1/60秒也就就是16.6ms//3.用这个间隔X0.8vsyncOffsetNs16.6ms*0.813.28msreturn snappedTimeNs - vsyncOffsetNs;}可以看到MediaCodecVideoRenderer参考positionUs时间和当前流的PTS进行时间同步保证同步。貌似目前还看不出和MediaCodecAudioRenderer 音频PTS的关系但可以肯定视频的PTS是参考其他时间进行同步的为了达到同步ExoPlayer用了大量的代码还考虑了程序的执行时间以纳秒级的计算尽量缩小了误差在极端情况下还会直接通过丢帧的方式保证同步这也就是有时候播放的文件解码压力比较大时视频会一卡一卡但是音频还是流畅播放的原因可以思考下为什么这么做反过来行不行。
参考时间戳的计算
MediaCodecVideoRenderer的参考positionUs在有音轨的情况下是通过MediaCodecAudioRenderer获取的MediaCodecAudioRenderer又通过调用DefaultAudiaSkink最终调用AudiaTrack的方法获取时间戳下面我们来看下具体获取的过程 private void updateCurrentPosition() {long newCurrentPositionUs audioSink.getCurrentPositionUs(isEnded());if (newCurrentPositionUs ! AudioSink.CURRENT_POSITION_NOT_SET) {currentPositionUs allowPositionDiscontinuity? newCurrentPositionUs: max(currentPositionUs, newCurrentPositionUs);allowPositionDiscontinuity false;}}//DefaultAudioSinkOverridepublic long getCurrentPositionUs(boolean sourceEnded) {if (!isAudioTrackInitialized() || startMediaTimeUsNeedsInit) {return CURRENT_POSITION_NOT_SET;}//主要从这里获取long positionUs audioTrackPositionTracker.getCurrentPositionUs(sourceEnded);//和通过帧数获取的时长位置取最小值positionUs min(positionUs, configuration.framesToDurationUs(getWrittenFrames()));return applySkipping(applyMediaPositionParameters(positionUs));}public long getCurrentPositionUs(boolean sourceEnded) {if (checkNotNull(this.audioTrack).getPlayState() PLAYSTATE_PLAYING) {//如果已经开始播放从AudiaTrack中同步出下面逻辑需要使用的数据以及获取smoothedPlayheadOffsetUs对getPlaybackHeadPositionUs做一个平滑处理maybeSampleSyncParams();}long systemTimeUs System.nanoTime() / 1000;long positionUs;AudioTimestampPoller audioTimestampPoller checkNotNull(this.audioTimestampPoller);boolean useGetTimestampMode audioTimestampPoller.hasAdvancingTimestamp();if (useGetTimestampMode) {//如果支持AudioTrack.getTimestamp优先使用// Calculate the speed-adjusted position using the timestamp (which may be in the future).long timestampPositionFrames audioTimestampPoller.getTimestampPositionFrames();//获取当前帧数long timestampPositionUs framesToDurationUs(timestampPositionFrames);//帧数转为时长long elapsedSinceTimestampUs systemTimeUs - audioTimestampPoller.getTimestampSystemTimeUs();elapsedSinceTimestampUs //计算和当前时间的差值Util.getMediaDurationForPlayoutDuration(elapsedSinceTimestampUs, audioTrackPlaybackSpeed);positionUs timestampPositionUs elapsedSinceTimestampUs;//当前的帧时长当前时间的差值当前位置} else {//否则使用getPlaybackHeadPositionUs的值if (playheadOffsetCount 0) {// 刚开始播放没有足够多的数据计算平滑差值直接取getPlaybackHeadPositionUspositionUs getPlaybackHeadPositionUs();} else {// getPlaybackHeadPositionUs() only has a granularity of ~20 ms, so we base the position off// the system clock (and a smoothed offset between it and the playhead position) so as to// prevent jitter in the reported positions.//AudiaTrack.getPlaybackHeadPositionUs获取的精度只有20ms,所以需要和当前时间的差值求一个平滑差值防止获取的getPlaybackHeadPositionUs有抖动positionUs Util.getMediaDurationForPlayoutDuration(systemTimeUs smoothedPlayheadOffsetUs, audioTrackPlaybackSpeed);}if (!sourceEnded) {//最终的位置还需要减去一个底层的延迟positionUs max(0, positionUs - latencyUs);}}if (lastSampleUsedGetTimestampMode ! useGetTimestampMode) {// 2次获取当前位置的方式不一样保存上一次的值previousModeSystemTimeUs lastSystemTimeUs;previousModePositionUs lastPositionUs;}long elapsedSincePreviousModeUs systemTimeUs - previousModeSystemTimeUs;//模式切换且和当前时间相差1s以内在1s内对上次的位置到当前时间做一个平滑过渡防止模式切换导致的跳动if (elapsedSincePreviousModeUs MODE_SWITCH_SMOOTHING_DURATION_US) {long previousModeProjectedPositionUs previousModePositionUs Util.getMediaDurationForPlayoutDuration(elapsedSincePreviousModeUs, audioTrackPlaybackSpeed);// 1s内取样1000次平滑过渡到当前时间long rampPoint (elapsedSincePreviousModeUs * 1000) / MODE_SWITCH_SMOOTHING_DURATION_US;positionUs * rampPoint;positionUs (1000 - rampPoint) * previousModeProjectedPositionUs;positionUs / 1000;}//需要监听位置首次增加的场景if (!notifiedPositionIncreasing positionUs lastPositionUs) {notifiedPositionIncreasing true;long mediaDurationSinceLastPositionUs Util.usToMs(positionUs - lastPositionUs);long playoutDurationSinceLastPositionUs Util.getPlayoutDurationForMediaDuration(mediaDurationSinceLastPositionUs, audioTrackPlaybackSpeed);long playoutStartSystemTimeMs //获取开始时间System.currentTimeMillis() - Util.usToMs(playoutDurationSinceLastPositionUs);listener.onPositionAdvancing(playoutStartSystemTimeMs);}lastSystemTimeUs systemTimeUs;lastPositionUs positionUs;lastSampleUsedGetTimestampMode useGetTimestampMode;return positionUs;}private void maybeSampleSyncParams() {//获取当前时间long systemTimeUs System.nanoTime() / 1000;//保证间隔30ms调用if (systemTimeUs - lastPlayheadSampleTimeUs MIN_PLAYHEAD_OFFSET_SAMPLE_INTERVAL_US) {//通过AudiaTrack.getPlaybackHeadPosition获取当前播放位置long playbackPositionUs getPlaybackHeadPositionUs();if (playbackPositionUs 0) {// 音频可能还未播放return;}// 最多取前10次playbackPositionUs 和当前时间的差值求出平均差值对playbackPositionUs 做一个平滑处理playheadOffsets[nextPlayheadOffsetIndex] //获取10次的差值存储Util.getPlayoutDurationForMediaDuration(playbackPositionUs, audioTrackPlaybackSpeed)- systemTimeUs;//每10次一个循环nextPlayheadOffsetIndex (nextPlayheadOffsetIndex 1) % MAX_PLAYHEAD_OFFSET_COUNT;if (playheadOffsetCount MAX_PLAYHEAD_OFFSET_COUNT) {playheadOffsetCount;}lastPlayheadSampleTimeUs systemTimeUs;smoothedPlayheadOffsetUs 0;//获取前几次差值的平均值获得平滑的差值后续通过当前时间这个值就可以计算出当前的PlaybackHeadPositionfor (int i 0; i playheadOffsetCount; i) {smoothedPlayheadOffsetUs playheadOffsets[i] / playheadOffsetCount;}}if (needsPassthroughWorkarounds) {//对于API 21/22的AC-3直出音轨后续获取的timestamp和latency 都是错误的值这里直接跳过return;}//audioTrack.getTimestamp获取timestampmaybePollAndCheckTimestamp(systemTimeUs);//audioTrack.getLatency获取底层的延迟maybeUpdateLatency(systemTimeUs);}private long getPlaybackHeadPositionUs() {return framesToDurationUs(getPlaybackHeadPosition());}private long getPlaybackHeadPosition() {//获取当前时间long currentTimeMs SystemClock.elapsedRealtime();if (stopTimestampUs ! C.TIME_UNSET) {//已经停止// Simulate the playback head position up to the total number of frames submitted.//获取当前到结束位置的时长long elapsedTimeSinceStopUs (currentTimeMs * 1000) - stopTimestampUs;//根据播放速度纠正时长long mediaTimeSinceStopUs Util.getMediaDurationForPlayoutDuration(elapsedTimeSinceStopUs, audioTrackPlaybackSpeed);//时长转帧数long framesSinceStop durationUsToFrames(mediaTimeSinceStopUs);//结束位置获取的总帧数结束位置到现在的帧数现在的总帧数再和结束位置以写入的总帧数取最小值return min(endPlaybackHeadPosition, stopPlaybackHeadPosition framesSinceStop);}//正常情况走下面逻辑保证间隔5ms调用一次if (currentTimeMs - lastRawPlaybackHeadPositionSampleTimeMs RAW_PLAYBACK_HEAD_POSITION_UPDATE_INTERVAL_MS) {updateRawPlaybackHeadPosition(currentTimeMs);lastRawPlaybackHeadPositionSampleTimeMs currentTimeMs;}return rawPlaybackHeadPosition (rawPlaybackHeadWrapCount 32);}private void updateRawPlaybackHeadPosition(long currentTimeMs) {AudioTrack audioTrack checkNotNull(this.audioTrack);int state audioTrack.getPlayState();if (state PLAYSTATE_STOPPED) {// The audio track hasnt been started. Keep initial zero timestamp.return;}//最终调用audioTrack.getPlaybackHeadPosition获取时长获取的为底层的无符号整型java中通过有符号的long来表示long rawPlaybackHeadPosition 0xFFFFFFFFL audioTrack.getPlaybackHeadPosition();if (needsPassthroughWorkarounds) {//这块是一个兼容处理对于API 21/22的直出音轨在暂停时获取到的值可能为0if (state PLAYSTATE_PAUSED rawPlaybackHeadPosition 0) {//保存为0时的位置passthroughWorkaroundPauseOffset this.rawPlaybackHeadPosition;}//这里进行恢复rawPlaybackHeadPosition passthroughWorkaroundPauseOffset;}if (Util.SDK_INT 29) {if (rawPlaybackHeadPosition 0 this.rawPlaybackHeadPosition 0 state PLAYSTATE_PLAYING) {//这段也是一个兼容处理当API29使用蓝牙设备播放时连接蓝牙失败时底层的状态已经停止但JAVA层的状态还是正在播放//当这种情况发生时获取位置为0if (forceResetWorkaroundTimeMs C.TIME_UNSET) {//通过设置这个标记来告诉当前处于错误状态当超过200ms后还是有问题会尝试重新初始化forceResetWorkaroundTimeMs currentTimeMs;}return;} else {forceResetWorkaroundTimeMs C.TIME_UNSET;}}if (this.rawPlaybackHeadPosition rawPlaybackHeadPosition) {// The value must have wrapped around.rawPlaybackHeadWrapCount;}this.rawPlaybackHeadPosition rawPlaybackHeadPosition;}//audioTrack.getTimestamp获取timestampprivate void maybePollAndCheckTimestamp(long systemTimeUs) {//audioTrack.getTimestamp不能平凡调用AudioTimestampPoller 是一个Audio Timestamp的轮询获取器稳定后控制调用者以10s的间隔去获取TimestampAudioTimestampPoller audioTimestampPoller checkNotNull(this.audioTimestampPoller);//是否获取到新的Timestamp条件是API必须大于等于19以支持这个函数且符合指定的时间间隔if (!audioTimestampPoller.maybePollTimestamp(systemTimeUs)) {return;}// 检验获取的Timestamplong audioTimestampSystemTimeUs audioTimestampPoller.getTimestampSystemTimeUs();long audioTimestampPositionFrames audioTimestampPoller.getTimestampPositionFrames();long playbackPositionUs getPlaybackHeadPositionUs();//不能和系统时间相差太大5sif (Math.abs(audioTimestampSystemTimeUs - systemTimeUs) MAX_AUDIO_TIMESTAMP_OFFSET_US) {listener.onSystemTimeUsMismatch(audioTimestampPositionFrames,audioTimestampSystemTimeUs,systemTimeUs,playbackPositionUs);audioTimestampPoller.rejectTimestamp();//不能和getPlaybackHeadPositionUs方法获取的值相差太大5s} else if (Math.abs(framesToDurationUs(audioTimestampPositionFrames) - playbackPositionUs) MAX_AUDIO_TIMESTAMP_OFFSET_US) {listener.onPositionFramesMismatch(audioTimestampPositionFrames,audioTimestampSystemTimeUs,systemTimeUs,playbackPositionUs);audioTimestampPoller.rejectTimestamp();} else {audioTimestampPoller.acceptTimestamp();}}//audioTrack.getTimestamp.getLatency获取底层的延迟private void maybeUpdateLatency(long systemTimeUs) {if (isOutputPcm//线性 PCM 编码 getLatencyMethod ! null//AudiaTreck存在getLatency方法 systemTimeUs - lastLatencySampleTimeUs MIN_LATENCY_SAMPLE_INTERVAL_US) {//调用间隔大于50mstry {//获取底层的延迟-bufferSizeUs排除缓冲区造成的延迟留下混音器和音频硬件驱动程序造成的延迟latencyUs castNonNull((Integer) getLatencyMethod.invoke(checkNotNull(audioTrack))) * 1000L- bufferSizeUs;// Check that the latency is non-negative.latencyUs max(latencyUs, 0);// Check that the latency isnt too large.if (latencyUs MAX_LATENCY_US) {listener.onInvalidLatency(latencyUs);latencyUs 0;}} catch (Exception e) {// The method existed, but doesnt work. Dont try again.getLatencyMethod null;}lastLatencySampleTimeUs systemTimeUs;}}Exoplayer 使用了2种方式来获取音轨的当前位置时间戳在API19及以上优先使用AudioTrack.getTimestamp来获取位置时间戳否则采用AudiaTrack.getPlaybackHeadPosition来获取由于getPlaybackHeadPosition精度较低还会采用一个平滑算法计算出一个平均值来优化getPlaybackHeadPosition的精度。
总结
Renderer作为一个重要的组件相比MediaSource的讲解可能比较简略一方面因为Renderer的整体结构比MediaSource简单没有分太多层代码也比较集中。但这并不意味着不重要这些代码值得仔细研究其实这短短代码中蕴含着开发人员无数次的尝试调优以及针对线上遇到问题的巧妙解决方案有些方案可能在我看来比较无奈当又比不可少。另一方面Renderer底层将解析工作交给了Android的系统组件如果想要追根溯源那又是另一个系列了。还有原因就是不能再写了网站的在线编辑器到这里每打一个字都要卡很久哈哈。 版权声明 © 本文为CSDN作者山雨楼原创文章 转载请注明出处 原创不易觉得有用的话收藏转发点赞支持
