H264 keyframe detection fail with correct RTSP stream from SharpRTSP

[Rakiah]

moved from: https://groups.google.com/forum/#!topic/meetecho-janus/ZTFWDyLvU6g
as this seems to be a bug

As stated in the google group discussion, receiving h264 streaming from a SharpRTSP generated RTSP stream doesn't work:

utils.c: line 643

gboolean janus_h264_is_keyframe(char* buffer, int len) {                                           
        JANUS_LOG(LOG_HUGE, "on before h264 keyframe\n");                                          
        if(!buffer || len < 0)                                                                     
                return FALSE;                                                                      
        JANUS_LOG(LOG_HUGE, "on after buffer check h264 keyframe\n");                              
        /* Parse H264 header now */                                                                
        uint8_t fragment = *buffer & 0x1F;                                                         
        uint8_t nal = *(buffer+1) & 0x1F;                                                          
        uint8_t start_bit = *(buffer+1) & 0x80;                                                    
        if(fragment == 5 ||                                                                        
                        ((fragment == 28 || fragment == 29) && nal == 5 && start_bit == 128)) {    
                JANUS_LOG(LOG_HUGE, "Got an H264 key frame\n");                                    
                return TRUE;                                                                       
        }                                                                                          
        JANUS_LOG(LOG_HUGE, "on after h264 is not keyframe\n");                                    
        /* If we got here it's not a key frame */                                                  
        return FALSE;                                                                              
}                                                                                                  

the return TRUE of this function is never called and therefore no keyframe is ever found, the stream however works correctly in VLC and other streaming solutions, which make me think that the Sharp RTSP stream is indeed correctly formatted but the header is wrongly parsed in this method ?

I'm trying to wrap my head around the specific part that parse the header but I do not manage to understand how the magic operations & numbers work

[lminiero]

Whether Janus detects a keyframe or not is not important: it's the browser that needs to detect something to decode, and if that's not happening it's a stream issue. Bringing VLC in the discussion is irrelevant, browsers have a different media stack and different constraints with respect to H264. Closing as not a bug.

[Rakiah]

I do think it is a bug since retrieving the camera directly without going through SharpRTSP actually works, and since SharpRTSP doesn't decode/reencode anything and just throw the frames untouched, why would this not work ? (also while I retrieve the camera directly without having to go through SharpRTSP, Janus actually detect key I-frames)
Can you please not close issues too fast and require first more informations ?
Thanks

@lminiero

[lminiero]

Janus just relays media as well, without transcoding.

[lminiero]

If it works without SharpRTSP in the middle, I'd say the problem is in SharpRTSP then.

[Rakiah]

That's what i would've said also, but the RTSP link provided by SharpRTSP is actually valid since it is playable by VLC and even webrtc-streamer, which is a similar solution to Janus

[Rakiah]

We can also see that the received SDP in our browser is definitely not the one we send with SharpRTSP, I believe Janus is indeed overriding some values, you spoke about a "videofmtp" attribute, is this maybe linked? @lminiero

[Rakiah]

Tracking down the problem,

it seems like the difference at browser side is that the fmtp end up beeing overriden, when we send the proxied stream from SharpRTSP to Janus the fmtp video is: 42e01f when using directly the camera Janus do not override the ftmp and leave 4d002a

[lminiero]

As I've said you can override the fmtp, check the videofmtp property which is explained in the configuration file.

[Rakiah]

@lminiero
Overriding the fmtp attribute didn't helped,

We've discussed a bit with the person behind SharpRTSP here:
ngraziano/SharpRTSP#53

Maybe something is relevant for you in the thread discussion to track down the problem ?

[REPRESSION]

Hi,
I enabled the webrtc logging, here are the logs:

https://pastebin.com/TJQ3Urfz

[22404:20404:1228/140614.833:WARNING:h264_sps_pps_tracker.cc(85)] No PPS with id << 0 received

this seems to be the problem for me.

here is my sdp:

v=0
o=user 123 0 IN IP4 0.0.0.0
s=SharpRTSP Test Camera
m=video 0 RTP/AVP 96
c=IN IP4 0.0.0.0
a=control:trackID=0
a=rtpmap:96 H264/90000
a=fmtp:96 profile-level-id=42A01E; sprop-parameter-sets=Z0IACvhgif+AAIAAiABB6wAM3+YEEA==,aM44gA==;

[RogerHardiman]

Just adding into this thread.
I wrote the RtspCameraExample which is in SharpRTSP.
I needed a cross platform H264 encoder so I used both TinyH264 and SimpleH264.
Both are H264 encoders which are lossless (ie they do not compress the video) and are very simple implementations of H264 (TinyH264 is about 20 lines of code).
It was done as a proof of concept. For real applications you would use a proper H264 encoder but I needed something I could run on iOS, Android, Mac, Windows and Linux.

So it is possible the problem is the very basic H264 encoder in the example.

Proper users of SharpRTSP would use a proper H264 encoder.

[REPRESSION]

I tried with h264 frames coming from an IPCamera and sending them into the rtsp stream (no TinyH264 in the loop), here is the result:
https://pastebin.com/7rVCuP43

The problem is different in this case:

[23660:23104:1228/154629.551:WARNING:video_receive_stream.cc(471)] No decodable frame in 200 ms, requesting keyframe.

the rtsp stream works in VLC