skipping segment: context deadline exceeded error

[camjac251]

Twitch might be making more changes to their ad injection, breaking compatibility even further. I ran a test of a few streamers yesterday and it gave back a few errors in the process.

Maybe it has to do with the extra functions I added master...camjac251:master

[maya] skipping segment: couldn't copy ts to output: read: context deadline exceeded (Client.Timeout or context cancellation while reading body)
[maya] skipping segment: got non-2xx http status 404 Not Found
[maya] stream over
[pokelawls] stream started
[pokelawls] recording to /mnt/twitch/streams/pokelawls/pokelawls-2020_12_24_07_33_24_UTC.ts
[pokelawls] skipping segment: couldn't copy ts to output: read: context deadline exceeded (Client.Timeout or context cancellation while reading body)
[pokelawls] skipping segment: got non-2xx http status 404 Not Found
[pokelawls] skipping segment: got non-2xx http status 404 Not Found
[pokelawls] skipping segment: couldn't copy ts to output: read: context deadline exceeded (Client.Timeout or context cancellation while reading body)
[pokelawls] skipping segment: got non-2xx http status 404 Not Found
[pokelawls] stream over
/mnt/twitch/streams/pokelawls/pokelawls-2020_12_24_07_33_24_UTC.ts
remuxing /mnt/twitch/streams/pokelawls/pokelawls-2020_12_24_07_33_24_UTC.ts to /mnt/twitch/streams/pokelawls/pokelawls-2020_12_24_07_33_24_UTC.mp4
jq: error (at <stdin>:1): Cannot index string with string "type"
parse error: Expected string key before ':' at line 1, column 7
>"type":"MESSAGE","data":{"topic":"video-playback-by-id.94753024","message":"{\"type\":\"viewcount\",\"server_time\":1608804480.298666,\"viewers\":5901}"}}
deleting /mnt/twitch/streams/pokelawls/pokelawls-2020_12_24_07_33_24_UTC.ts
[pokelawls] stream started
[pokelawls] recording to /mnt/twitch/streams/pokelawls/pokelawls-2020_12_24_10_08_50_UTC.ts
[pokelawls] could not extract playlist: stream is offline
[pokelawls] output file /mnt/twitch/streams/pokelawls/pokelawls-2020_12_24_10_08_50_UTC.ts was empty, removing...

There is also a jq error that happened a few times but is likely unrelated

jq: error (at <stdin>:1): Cannot index string with string "type"
parse error: Expected string key before ':' at line 1, column 7
>"type":"MESSAGE","data":{"topic":"video-playback-by-id.94753024","message":"{\"type\":\"viewcount\",\"server_time\":1608786691.129787,\"viewers\":5335}"}}
jq: error (at <stdin>:1): Cannot index string with string "type"
parse error: Expected string key before ':' at line 1, column 7
>"type":"MESSAGE","data":{"topic":"video-playback-by-id.96858382","message":"{\"type\":\"viewcount\",\"server_time\":1608786691.129787,\"viewers\":2303}"}}
jq: error (at <stdin>:1): Cannot index string with string "type"
parse error: Expected string key before ':' at line 1, column 7
>"type":"MESSAGE","data":{"topic":"video-playback-by-id.38746172","message":"{\"type\":\"viewcount\",\"server_time\":1608786691.129787,\"viewers\":16067}"}}
jq: error (at <stdin>:1): Cannot index string with string "type"
parse error: Expected string key before ':' at line 1, column 7
>"type":"MESSAGE","data":{"topic":"video-playback-by-id.96858382","message":"{\"type\":\"viewcount\",\"server_time\":1608786721.245259,\"viewers\":2326}"}}
jq: error (at <stdin>:1): Cannot index string with string "type"
parse error: Expected string key before ':' at line 1, column 7
>"type":"MESSAGE","data":{"topic":"video-playback-by-id.94753024","message":"{\"type\":\"viewcount\",\"server_time\":1608786721.245259,\"viewers\":5477}"}}
jq: error (at <stdin>:1): Cannot index string with string "type"
parse error: Expected string key before ':' at line 1, column 7
>"type":"MESSAGE","data":{"topic":"video-playback-by-id.247808909","message":"{\"type\":\"viewcount\",\"server_time\":1608787981.529022,\"viewers\":1502}"}}
jq: error (at <stdin>:1): Cannot index string with string "type"
parse error: Expected string key before ':' at line 1, column 7
>"type":"MESSAGE","data":{"topic":"video-playback-by-id.94753024","message":"{\"type\":\"viewcount\",\"server_time\":1608789451.299460,\"viewers\":8093}"}}

[Hakkin]

A few questions:
Did the whole recording fail for these streams, or just a select few segments?
Can you find associated VODs for them on Twitch?
How long of a period were these recordings over?
The timeout on segments is 10 seconds, which is quite a long time to hang on a segment, considering Twitch video segments are normally 2 seconds long.
Timeouts can be caused by multiple things, including issues with the streamer's internet, issues with Twitch's servers, and issues with your own internet.
There's really no way to get around these timeouts while still preserving the "streaming" nature of twitchpipe, since we can't wait on segments to download all day, the segments in front of it will eventually expire due to the live nature of the stream.

The jq error is interesting, I'm not really sure what to make of it. It seems like Twitch may be sending malformed JSON payloads, definitely something to look into since we wouldn't want to miss the initial stream starting announcement.