Letting user know when a packet has dropped

[jrsnen]

It would be nice if uvgRTP would tell when a packet was lost during transit. Currently, the user can have a very hard time determining if a frame was lost.

Something like a callback function would probably be useful.

The RTCP reports are not specific enough to determine which packet was lost.

[altonen]

There was a plan to implement this at some point but then we pivoted to having the frame receiver drop inter frames until it received an intra so I never finished the implementation but some remnants of it still remain:
https://github.com/ultravideo/uvgRTP/blob/master/src/media_stream.cc#L472
https://github.com/ultravideo/uvgRTP/blob/master/include/uvgrtp/util.hh#L284

[jrsnen]

@altonen Is there something I can do to detect missing frames to drop those before next Intra? I'm currently trying to get rid of gray screens caused by dropped frames (currently, mostly from #105), but I have not been able to find a way (other than maybe POC in the frame itself) to determine which frames were lost.

I was under the impression that the frame dropping would be done on the application side and that is why the notify-callback was needed.

[altonen]

Can't you determine which frames were lost based on which frames were garbage collected? Maybe I'm not understanding the question but I think determining which frames are lost is quite easy because they are still incomplete in the frame buffer when their timer expire.

[jrsnen]

Oh, maybe I'm misremembering. That would probably be ok, if it could be disabled via flags and this notify would be implemented for those who want to self-manage.

As it is, neither is usable, so if I understand correctly, it is currently not possible to react to lost frames to avoid those gray screens?

I created a new issue for this (#121).

[altonen]

I think you can disable the behavior by giving RCE_H26X_NO_INTRA_DELAY in which case uvgRTP doesn't drop inters/wait for intras but instead returns all received inter frames, regardless of whether the intra of that GOP(?) was received. Or at least that's how it's suppose to work.

Currently there's no way for the application to react to, or even know about, if/when a frame has been dropped.

[jrsnen]

A system inside uvgRTP is probably better than having the user to react to lost frames (I already made a simple version). Closing this issue. If someone has need for this, they can reopen this or create a new issue.