Thumbnail time is completly wrong.

[edemir206]

Hi,

I've been trying to use your solution, but for some reason thumbnails "time" seems to be messed up. I've been trying to import the thumbs using clappr thumbnails plugin using the following code:

function addThumbs()
{

var request = $.ajax({
	  url: "/app/myapp/thumbs/<?php echo get_field('stream') ?>/thumbnails.json",
	  type: "GET",
	  dataType: "json"
	});

	request.done(function(retorno) {

		$.each( retorno.segments, function( key, val ) {
				var thumbnailsPlugin = player.getPlugin("scrub-thumbnails");
				var newThumb = {
					url: "/app/myapp/thumbs/<?php echo get_field('stream') ?>/"+val.thumbnails[0].name,
					time: val.thumbnails[0].time
				};
				
				thumbnailsPlugin.addThumbnail(newThumb).then(function() {
					console.log("Thumbnail added."+newThumb.url+" Time: "+val.thumbnails[0].time);
				});
		});
	});
}

setInterval(function(){ addThumbs(); }, 15000); 

addThumbs();

This is part of my generated thumbnails.json:

"segments":[{"sn":0,"thumbnails":[{"time":0,"name":"283ec_recordall-0-0.jpg"}]},{"sn":1,"thumbnails":[{"time":12.134,"name":"283ec_recordall-1-0.jpg"}]},{"sn":2,"thumbnails":[{"time":3.432000000000002,"name":"283ec_recordall-2-0.jpg"}]},{"sn":4,"thumbnails":[{"time":0,"name":"283ec_recordall-4-0.jpg"}]},

Well, as you can see the time is not continuous, it start with 0, then 12 and the next is 3.43.

I'm using NGINX + RTMP plugin to generate the HLS and pointing the .m3u8 as intended.

In my nginx application the hls_fragment is set to 10s

These are the options (in nginx.conf) i'm using to generate the thumbs:
exec bash -c "hls-live-thumbnails http://nginx/app/hls/$name/index.m3u8 --interval 30 --targetThumbnailCount 30 --outputDir /data/app/thumbs/$name --outputNamePrefix $name";

What am I doing wrong ? Am I misunderstanding how your plugin works ?

[tjenkinson]

@edemir206 the time is the time into the segment the thumbnail was generated from. E.g. If you know that each of your segments is exactly 10 seconds long then to get the real time you would add sn*10

The reason they are relative to the segment is because for a live stream there is no fixed starting point.

Let me know if you get something working with Clappr. I was actually planning on updating the thumbnails plugin to support this but never got around to it.

[edemir206]

Hi @tjenkinson

I understand now. But even using sn*10 in my case the thumbs won't get synced correcly, the video position won't translate the thumb position, there must be something I'm missing.

Below is my updated code:

var player = new Clappr.Player({
source: "/app/farol/hls/<?php echo get_field('stream') ?>/index.m3u8", 
parentId: "#video",
autoPlay: true,
  plugins: {
    core: [ClapprThumbnailsPlugin]
  },
  scrubThumbnails: {
    backdropHeight: 64,
    spotlightHeight: 84,
    thumbs: []
  }
});


var added_thumbs = [];

function addThumbs()
{

	var request = $.ajax({
	  url: "/app/farol/thumbs/<?php echo get_field('stream') ?>/thumbnails.json",
	  type: "GET",
	  dataType: "json"
	});

	request.done(function(retorno) {
		$.each( retorno.segments, function( key, val ) {
			if(added_thumbs.indexOf(val.sn) === -1)
			{
				$.each( val.thumbnails, function( thumb_key, thumb_val ) {				
					var thumbnailsPlugin = player.getPlugin("scrub-thumbnails");
					var newThumb = {
						url: "/app/farol/thumbs/<?php echo get_field('stream') ?>/"+thumb_val.name,
						time: val.sn*10+thumb_val.time
					};
					
					thumbnailsPlugin.addThumbnail(newThumb).then(function() {
						console.log("Thumbnail added."+newThumb.url+" Time: "+thumb_val.time);
					});
				});
				added_thumbs.push(val.sn);
			}
		});
	});
}

setInterval(function(){ addThumbs(); }, 15000); 

addThumbs();

[tjenkinson]

@edemir206 what type of stream is it? LIVE, VOD or EVENT? If all the segments are 10 seconds and it is VOD or EVENT it should work. It would be more accurate though, and work for LIVE, to get the segment start times from hls.js/clappr.

[edemir206]

@tjenkinson it is a LIVE stream.

I'm using the exec_push nginx rtmp directive to convert the stream to a lower bitrate using ffmpeg, below is my nginx application

        application transmissao{
            # record all;

            exec_push ffmpeg -i rtmp://farol:1935/transmissao/$name -async 1 -vsync -1 -c:v libx264 -c:a libfdk_aac -profile:a aac_he_v2 -vbr 3 -tune zerolatency -crf 23 -maxrate 2M -bufsize 4M -f flv rtmp://farol:1935/hls/$name;
            #SCRIPTS DE AUTENTICACAO E DE CONEXAO
            on_publish    nginx:80/farol?farol_publish;
            on_publish_done    nginx:80/farol?farol_publish_done;

            #THUMB GENERATION
            exec bash -c "hls-live-thumbnails http://nginx/app/farol/hls/$name/index.m3u8 --interval 45 --outputDir /data/farol/thumbs/$name --outputNamePrefix $name";
        }

        application hls{
            hls on;
            hls_playlist_length 24h;
            hls_fragment 10s;
            # record all;
        }

[tjenkinson]

Ah right. Yes you will need more logic in the player to get the start time of the segments in the media element (MSE buffer).

The following can happen:

• Player requests playlist
• the first segment in the playlist is sequence number 5 (because the first few were removed already)
• player starts by appending segment with sequence number 8 (because this is closer to the live point).

This means currentTime 0 in the media element maps to the start time of segment 8

[edemir206]

@tjenkinson I followed your instructions, started from segment 8, but even this way there's a problem with time syncing. The thumbs represets video time up to 10 min, but after that everything gets messed up that's really weird. I don't know what else to do I think i'm just gonna give up.