Emotes from twitch subscriptions are not loading

[iwsfg]

Hi. I am subscribed to an unreasonable number of channels and it appears that Dank Chat can't handle this many emotes?

The "Subs" tab of the emote picker is always empty, and autocomplete only suggests me emotes from BTTV/FFZ/7TV that are enabled for the channel. Unless I remember emote names exactly and type them in from the keyboard in full there is no way for me to post them. And doing it like this is rather difficult and limiting.

How can I help you debug it?

I'm seeing this on two devices:

• Android 11 with the app v3.8.7, and
• Android 13 with the app v3.7.4

edit: fixed my broken english

[flex3r]

Are you still experiencing this issue or was it only a temporary hiccup?

[iwsfg]

No, this issue is persistent. I have been experiencing it ever since I started using the app around a month ago, and I'm still seeing it now on both devices that I mentioned in the opening post. Force reloading emotes from the ⋮ > More > Reload emotes menu doesn't help it

[flex3r]

Hm, very weird. Since you can reproduce it on different devices it has to be an issue with the way I load emote set information.
Is it possible for you to access DankChat's Logcat output? This can either be done directly via adb or via a dedicated Logcat app on your device.
With the logs we might be able to pinpoint what step fails in your case.

[iwsfg]

Sure, here's me switching into a force-stopped app and then hitting More > Reload emotes once more when I thought it should have finished loading.

logcat_11-24-2023_21-20-32.txt

I don't know it if it's the log entry that cuts off early or if the actual URLs are like this, logs were grabbed with com.dp.logcatapp from F-Droid if that matters

tl;dr: it's trying to fetch emotes for all subscriptions in one go and gets 414 - URI Too Long error

[flex3r]

Welp, that explains it. I'll try to find some time for a quick fix this weekend.

[iwsfg]

Thank you! ♥️

[flex3r]

I pushed a simple fix, would you be able to test this relase if it fixes your issue?
https://github.com/flex3r/DankChat/releases/tag/release

[iwsfg]

Thank you so much for fixing it!

Works almost flawlessly, but some emote packs were strangely missing until I opened a new chat tab. And I think some emotes within the packs weren't showing either, not even the circle placeholder for them.

I'll try to describe the exact steps I've taken:

• It didn't let me install downloaded apk over version from Google Play without uninstalling first, so I did that.
• On a newly installed copy of the app I opened a new (first) chat tab. It was a currently offline channel and chat was empty.
• Waited for spinner to finish spinning, then pressed Login from three dots menu.
• Once I got to the web view, task switched into password manager app, then back into the DankChat;
• signed into Twitch, returned from webview and waited for it to finish loading once again.
• Looked at the emotes via the emote picker menu, tried getting autocomplete to suggest me a couple of them without sending any messages
• Noticed some emotes were missing from the autocomplete menu, actually whole packs by user prefixes

I then tried manually counting number of channels in the emote picker and it turned 9 packs short, curiously, that's exactly how many gifted subscriptions I currently have but it wasn't exclusively the gifted subs that didn't load. Some of the gifted subs were available while others weren't. Or at least they didn't make it to the UI and autocomplete logic.

• I then tried opening a new tab with my own channel, intending to type one in from the keyboard in full
• Started typing one of the emotes that previously wasn't autocompleting
• It showed up in the autocomplete

At that point I once again manually re-counted packs in the Subs tab of the emote picker and got the right amount that matches the number of active subscriptions I see on the twitch.tv/subscriptions page (including gifted)

I did not have the logcat running during this. Can attempt to reproduce it tomorrow if needed.

Oh, and one of the packs that was only displaying 2 out of 11 emotes at first, without placeholders or reserved space for the rest, was subscription to anoravt (product_id 401420304, user id 812602212), in case you keep logs on the server and want to check those.

[flex3r]

Thanks for the detailed reply!
I unfortunately do not store any logs so a logcat reproducer would definitely help.

First some background how the Twitch emote resolution currently works:
All available Twitch emote sets are sent as ids via a specific IRC message.
Because there is no good official API from Twitch to load emote set details by id, I am using a 3rd party api provided by someone in the Chatterino community.

Since you at least seem to have some emote sets showing up it seems like only some of the emote set resolution failed.
My current assumption is that your amount of available emotes might have caused timeouts in the 3rd party API internally. But this should not be consecutively reproducable since all sets would be cached at some point.
On the other hand it could be that the http client used by DankChat ran into a request timeout. Not sure, logcat would definitely help here.

As to why it fixed itself after adding a channel, I checked the data loading logic and it seems like when adding a channel, most of the loading steps are performed again. (Some of these are definitely unnecessary, like global 3rd party emotes - Added a TODO for myself to optimize that)
So as a side effect, adding a new channel caused all twitch emote sets to be loaded again and succeeded that time..

Regarding your second issue, I used your initial uploaded logcat and saw that the channel #anoravt actually has several different emote sets.
The first one being 354525835, containing only two animated emotes. The other one being 375231614:
https://flxrs.com/api/sets?id=354525835,375231614
It could very well be that one of the ids was part of the request that failed, while the other one suceeded.
At least that is my current assumption.

[iwsfg]

Thank you for a such detailed write up as well, it was a huge help. I don't usually deal with Android or Kotlin, so I struggled reading the code a little.

I looked at anora's emotes a bit more and noticed that there's a third set for follower emotes, which surprisingly uses UUID for id instead of a numeric ID like the rest of them. And it's the one that contains two emotes I saw, so actually neither of the other two were loaded on the first try. Follower emotes are also made globally available with the subscription since about a year ago.

I didn't realize there more than one emote set per channel, that explains a lot. Definitely answers how I was able to count most of the channels in the emote picker, even though some of the chunks failed to load.

Server error as you suggested, either in your backend or on that API's side, seems to be the most likely cause. And if those sets are temporarily cached on one of the two servers involved, that would also explain why it managed to go through on the second try. Client side logic seems sound enough to not be thrown off by 36-characters long ids.

Will definitely try to reproduce it tomorrow. Hopefully caches are cold by then and it will be easy to trigger.