[meta-freescale] imx-vpu-hantro: broken A/V sync with VP8 playback

[Carlos Rafael Giani]

Hi,

I forgot to answer this after further investigations.

It turns out that the frames that are output by the codec interface (in 
case of VP8) are _only_ those who can be displayed. So, if decode() 
returns CODEC_HAS_FRAME, but a subsequent getframe() call doesn't, then 
the last encoded VP8 frame that was input into the codec was an 
invisible frame.

It is still unclear though if this affects VP9 playback as well.

On 13.12.18 14:48, Otavio Salvador wrote:
> Hello Carlos,
>
> On Sat, Dec 8, 2018 at 8:43 AM Carlos Rafael Giani <crg7475 at mailbox.org> wrote:
>> I recently noticed that VP8 playback does not work correctly with
>> imx-vpu-hantro . If a VP8 stream contains alternate reference (altref)
>> frames, then imx-vpu-hantro decodes them correctly, but does not inform
>> about the fact that they are altref frames. This is a problem, because
>> altref frames are supposed to be decoded, and NOT shown. In particular,
>> timestamps should not be increased because of a decoded altref frame.
>> They essentially are for internal use only.
>>
>> However, because the callers do not know that a frame is an altref one,
>> they don't skip them like they should. As a result, A/V sync gradually
>> gets worse over time.
>>
>> Is there any newer imx-vpu-hantro version that fixes this? The necessary
>> fields are already there. Specifically, there's isGoldenOrAlternate in
>> the FRAME struct in openmax_il/source/decoder/codec.h . However, it is
>> never set to TRUE . I did some digging, and the only part that sets any
>> flag like that is in vp8decmcapi.c (which is not used), and even then,
>> this concerns itself only with golden frames, not altref ones.
> Adding Carol to Cc...
>