Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(src/es): Improvements for the main spanish extensions (part 2) #2474

Closed
wants to merge 6 commits into from
Closed

refactor(src/es): Improvements for the main spanish extensions (part 2) #2474

wants to merge 6 commits into from

Conversation

imper1aldev
Copy link
Contributor

Checklist:

Closes #2392
Closes #2407
Closes #2409
Closes #2410
Closes #2362

  • Updated extVersionCode value in build.gradle for individual extensions
  • Updated overrideVersionCode or baseVersionCode as needed for all multisrc extensions
  • Referenced all related issues in the PR body (e.g. "Closes #xyz")
  • Added the isNsfw = true flag in build.gradle when appropriate
  • Have not changed source names
  • Have explicitly kept the id if a source's name or language were changed
  • Have tested the modifications by compiling and running the extension through Android Studio

episode_number = formatedEp
name = "Episodio $formatedEp"
}
}.reversed()
}

override fun videoListParse(response: Response): List<Video> {
val document = response.asJsoup()
val videoList = mutableListOf<Video>()
document.select("div.playermain ul.dropcaps li#play-video a.cap").forEach { players ->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if you're using this many extractors, you should parallelize this loop.
Same for the other sources in this loop.

Copy link
Contributor

@Claudemirovsky Claudemirovsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we live in a society

Comment on lines +41 to 42
override val name = "DoramasYT"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Have explicitly kept the id if a source's name or language were changed
Suggested change
override val name = "DoramasYT"
override val name = "DoramasYT"
override val id = 885919523581666049L

Comment on lines +71 to +92
override fun popularAnimeParse(response: Response): AnimesPage {
val url = response.request.url.toString()
val document = response.asJsoup()
val elements = document.select("div.col-lg-2.col-md-4.col-6 div.animes")
val nextPage = document.select("ul.pagination li:last-child a").any()
val animeList = elements.map { element ->
SAnime.create().apply {
setUrlWithoutDomain(
when {
url.contains("emision") -> element.select("a").attr("abs:href")
else -> element.select("div.anithumb a").attr("abs:href")
},
)
title = element.select("div.animedtls p").text()
description = element.select("div.animedtls p").text()
thumbnail_url = when {
url.contains("emision") -> element.select("a > img").attr("abs:src")
else -> element.select(".anithumb a img").attr("abs:src")
}
}
}
return AnimesPage(animeList, nextPage)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you're going to use Jsoup(document selectors and etc), then the source class should be a ParsedAnimeHttpSource and you should use it's jsoup-related functions, in this case it would look like this:

    override fun popularAnimeSelector() = "div.col-lg-2.col-md-4.col-6 div.animes"

    override fun popularAnimeFromElement(element: Element) = SAnime.create().apply {
        val urlElement = element.selectFirst("a")!!
        setUrlWithoutDomain(urlElement.attr("href"))
        title = element.selectFirst("div.animedtls p")!!.text()
        thumbnail_url = urlElement.selectFirst("img")?.absUrl("src")
    }
    
    override fun popularAnimeNextPageSelector() = "ul.pagination li:last-child a"

override fun episodeListParse(response: Response): List<SEpisode> {
return super.episodeListParse(response).reversed()
}
override fun latestUpdatesParse(response: Response) = popularAnimeParse(response)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

^ this also would work with the previous example

Comment on lines +109 to +122
override fun searchAnimeParse(response: Response): AnimesPage {
val document = response.asJsoup()
val elements = document.select("div.col-lg-2.col-md-4.col-6 div.animes")
val nextPage = document.select("ul.pagination li:last-child a").any()
val animeList = elements.map { element ->
SAnime.create().apply {
setUrlWithoutDomain(element.selectFirst("a")!!.attr("href"))
title = element.select("div.animedtls p").text()
thumbnail_url = element.select("a img").attr("src")
description = element.select("div.animedtls p").text()
}
}
episode.setUrlWithoutDomain(element.select("a").attr("href"))
episode.episode_number = formatedEp
episode.name = "Episodio $formatedEp"
return AnimesPage(animeList, nextPage)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's pratically the same code you wrote in popularAnimeParse(response); why not just reuse it?
(also, this version would work with popular/latest anime pages just fine too)

else -> 1F
}
SEpisode.create().apply {
setUrlWithoutDomain(element.select("a").attr("abs:href"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you don't need the absolute url, setUrlWithoutDomain will save only the path (as it's name says).
Ex: https://doramayt.com/ihatetheantichrist -> /ihatetheantichrist

Comment on lines +168 to 236
if (embedUrl.contains("voe")) {
VoeExtractor(client).videoFromUrl(url, prefix = "Voe:")?.let { videoList.add(it) }
}
if ((embedUrl.contains("amazon") || embedUrl.contains("amz")) && !embedUrl.contains("disable")) {
val video = amazonExtractor(baseUrl + url.substringAfter(".."))
if (video.isNotBlank()) {
if (url.contains("&ext=es")) {
videoList.add(Video(video, "AmazonES", video))
} else {
videoList.add(Video(video, "Amazon", video))
}
}
}
if (server.contains("ok")) {
val videos = OkruExtractor(client).videosFromUrl(url)
videoList.addAll(videos)
if (embedUrl.contains("ok.ru") || embedUrl.contains("okru")) {
runCatching {
OkruExtractor(client).videosFromUrl(url).also(videoList::addAll)
}
}
if (embedUrl.contains("filemoon") || embedUrl.contains("moonplayer")) {
val vidHeaders = headers.newBuilder()
.add("Origin", "https://${url.toHttpUrl().host}")
.add("Referer", "https://${url.toHttpUrl().host}/")
.build()
FilemoonExtractor(client).videosFromUrl(url, prefix = "Filemoon:", headers = vidHeaders).also(videoList::addAll)
}
if (embedUrl.contains("uqload") || embedUrl.contains("upload")) {
UqloadExtractor(client).videosFromUrl(url).also(videoList::addAll)
}
if (embedUrl.contains("mp4upload")) {
Mp4uploadExtractor(client).videosFromUrl(url, headers).let { videoList.addAll(it) }
}
if (embedUrl.contains("wishembed") || embedUrl.contains("embedwish") || embedUrl.contains("streamwish") || embedUrl.contains("strwish") || embedUrl.contains("wish")) {
val docHeaders = headers.newBuilder()
.add("Origin", "https://streamwish.to")
.add("Referer", "https://streamwish.to/")
.build()
StreamWishExtractor(client, docHeaders).videosFromUrl(url, videoNameGen = { "StreamWish:$it" }).also(videoList::addAll)
}
if (embedUrl.contains("doodstream") || embedUrl.contains("dood.")) {
DoodExtractor(client).videoFromUrl(url, "DoodStream", false)?.let { videoList.add(it) }
}
if (embedUrl.contains("streamlare")) {
StreamlareExtractor(client).videosFromUrl(url).let { videoList.addAll(it) }
}
if (embedUrl.contains("yourupload") || embedUrl.contains("upload")) {
YourUploadExtractor(client).videoFromUrl(url, headers = headers).let { videoList.addAll(it) }
}
if (embedUrl.contains("burstcloud") || embedUrl.contains("burst")) {
BurstCloudExtractor(client).videoFromUrl(url, headers = headers).let { videoList.addAll(it) }
}
if (embedUrl.contains("fastream")) {
FastreamExtractor(client, headers).videosFromUrl(url).also(videoList::addAll)
}
if (embedUrl.contains("upstream")) {
UpstreamExtractor(client).videosFromUrl(url).let { videoList.addAll(it) }
}
if (embedUrl.contains("streamtape") || embedUrl.contains("stp") || embedUrl.contains("stape")) {
StreamTapeExtractor(client).videoFromUrl(url)?.let { videoList.add(it) }
}
if (server.contains("zeus")) {
if (embedUrl.contains("ahvsh") || embedUrl.contains("streamhide")) {
StreamHideVidExtractor(client).videosFromUrl(url).let { videoList.addAll(it) }
}
if (embedUrl.contains("filelions") || embedUrl.contains("lion")) {
StreamWishExtractor(client, headers).videosFromUrl(url, videoNameGen = { "FileLions:$it" }).also(videoList::addAll)
}
if (embedUrl.contains("solid")) {
val videos = SolidFilesExtractor(client).videosFromUrl(url)
videoList.addAll(videos)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A URL won't work on more than one extractor, so you should be using else ifs instead of only ifs.
Personally I'd recommend using a when {} block, but that's up to you.

Comment on lines +145 to +210
if (embedUrl.contains("voe")) {
VoeExtractor(client).videoFromUrl(url, prefix = "Voe:")?.let { videoList.add(it) }
}
if (embedUrl.contains("streamlare")) {
try {
StreamlareExtractor(client).videosFromUrl(url)?.let {
videoList.add(it)
if ((embedUrl.contains("amazon") || embedUrl.contains("amz")) && !embedUrl.contains("disable")) {
val video = amazonExtractor(baseUrl + url.substringAfter(".."))
if (video.isNotBlank()) {
if (url.contains("&ext=es")) {
videoList.add(Video(video, "AmazonES", video))
} else {
videoList.add(Video(video, "Amazon", video))
}
} catch (_: Exception) {}
}
if (embedUrl.contains("doodstream") || embedUrl.contains("dood")) {
val video = try {
DoodExtractor(client).videoFromUrl(url, "DoodStream", true)
} catch (e: Exception) {
null
}
if (video != null) {
videoList.add(video)
}
if (embedUrl.contains("ok.ru") || embedUrl.contains("okru")) {
runCatching {
OkruExtractor(client).videosFromUrl(url).also(videoList::addAll)
}
}
if (embedUrl.contains("okru") || embedUrl.contains("ok.ru")) {
val videos = OkruExtractor(client).videosFromUrl(url)
videoList.addAll(videos)
if (embedUrl.contains("filemoon") || embedUrl.contains("moonplayer")) {
val vidHeaders = headers.newBuilder()
.add("Origin", "https://${url.toHttpUrl().host}")
.add("Referer", "https://${url.toHttpUrl().host}/")
.build()
FilemoonExtractor(client).videosFromUrl(url, prefix = "Filemoon:", headers = vidHeaders).also(videoList::addAll)
}
}
if (embedUrl.contains("uqload") || embedUrl.contains("upload")) {
UqloadExtractor(client).videosFromUrl(url).also(videoList::addAll)
}
if (embedUrl.contains("mp4upload")) {
Mp4uploadExtractor(client).videosFromUrl(url, headers).let { videoList.addAll(it) }
}
if (embedUrl.contains("wishembed") || embedUrl.contains("embedwish") || embedUrl.contains("streamwish") || embedUrl.contains("strwish") || embedUrl.contains("wish")) {
val docHeaders = headers.newBuilder()
.add("Origin", "https://streamwish.to")
.add("Referer", "https://streamwish.to/")
.build()
StreamWishExtractor(client, docHeaders).videosFromUrl(url, videoNameGen = { "StreamWish:$it" }).also(videoList::addAll)
}
if (embedUrl.contains("doodstream") || embedUrl.contains("dood.")) {
DoodExtractor(client).videoFromUrl(url, "DoodStream", false)?.let { videoList.add(it) }
}
if (embedUrl.contains("streamlare")) {
StreamlareExtractor(client).videosFromUrl(url).let { videoList.addAll(it) }
}
if (embedUrl.contains("yourupload") || embedUrl.contains("upload")) {
YourUploadExtractor(client).videoFromUrl(url, headers = headers).let { videoList.addAll(it) }
}
if (embedUrl.contains("burstcloud") || embedUrl.contains("burst")) {
BurstCloudExtractor(client).videoFromUrl(url, headers = headers).let { videoList.addAll(it) }
}
if (embedUrl.contains("fastream")) {
FastreamExtractor(client, headers).videosFromUrl(url).also(videoList::addAll)
}
if (embedUrl.contains("upstream")) {
UpstreamExtractor(client).videosFromUrl(url).let { videoList.addAll(it) }
}
if (embedUrl.contains("streamtape") || embedUrl.contains("stp") || embedUrl.contains("stape")) {
StreamTapeExtractor(client).videoFromUrl(url)?.let { videoList.add(it) }
}
if (embedUrl.contains("ahvsh") || embedUrl.contains("streamhide")) {
StreamHideVidExtractor(client).videosFromUrl(url).let { videoList.addAll(it) }
}
if (embedUrl.contains("filelions") || embedUrl.contains("lion")) {
StreamWishExtractor(client, headers).videosFromUrl(url, videoNameGen = { "FileLions:$it" }).also(videoList::addAll)
}
} catch (_: Exception) { }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As said before, use else ifs or a when {} block instead of a trillion open ifs.
Also, as jmir said, you should try parallelizing the extractor, because loading this all wiill take decades.

Comment on lines +272 to +282
document.select(".mvic-info .mvici-left p").map {
if (it.select("strong").text().contains("Genre", true)) {
anime.genre = it.select("a").joinToString { it.text() }
}
if (it.select("strong").text().contains("Actors", true)) {
anime.artist = it.selectFirst("a")?.text() ?: ""
}
if (it.select("strong").text().contains("Studio", true)) {
anime.author = it.selectFirst("a")?.text() ?: ""
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
document.select(".mvic-info .mvici-left p").map {
if (it.select("strong").text().contains("Genre", true)) {
anime.genre = it.select("a").joinToString { it.text() }
}
if (it.select("strong").text().contains("Actors", true)) {
anime.artist = it.selectFirst("a")?.text() ?: ""
}
if (it.select("strong").text().contains("Studio", true)) {
anime.author = it.selectFirst("a")?.text() ?: ""
}
}
document.select(".mvic-info .mvici-left p").forEach {
val text = it.selectFirst("strong")?.text() ?: return@forEach
when {
text.contains("Genre", true) -> anime.genre = it.select("a").joinToString { it.text() }
text.contains("Actors", true) -> anime.artist = it.selectFirst("a")?.text()
text.contains("Studio", true) -> anime.author = it.selectFirst("a")?.text()
}
}

Comment on lines +313 to +380
if (embedUrl.contains("voe")) {
VoeExtractor(client).videoFromUrl(url, prefix = "Voe:")?.let { videoList.add(it) }
}
if ((embedUrl.contains("amazon") || embedUrl.contains("amz")) && !embedUrl.contains("disable")) {
val video = amazonExtractor(baseUrl + url.substringAfter(".."))
if (video.isNotBlank()) {
if (url.contains("&ext=es")) {
videoList.add(Video(video, "AmazonES", video))
} else {
videoList.add(Video(video, "Amazon", video))
}
}
url.contains("yourupload") -> {
val videos = YourUploadExtractor(client).videoFromUrl(url, headers = headers, name = "Original", prefix = prefix)
videoList.addAll(videos)
}
if (embedUrl.contains("ok.ru") || embedUrl.contains("okru")) {
runCatching {
OkruExtractor(client).videosFromUrl(url).also(videoList::addAll)
}
}
}

return videoList.sort()
if (embedUrl.contains("filemoon") || embedUrl.contains("moonplayer")) {
val vidHeaders = headers.newBuilder()
.add("Origin", "https://${url.toHttpUrl().host}")
.add("Referer", "https://${url.toHttpUrl().host}/")
.build()
FilemoonExtractor(client).videosFromUrl(url, prefix = "Filemoon:", headers = vidHeaders).also(videoList::addAll)
}
if (embedUrl.contains("uqload") || embedUrl.contains("upload")) {
UqloadExtractor(client).videosFromUrl(url).also(videoList::addAll)
}
if (embedUrl.contains("mp4upload")) {
Mp4uploadExtractor(client).videosFromUrl(url, headers).let { videoList.addAll(it) }
}
if (embedUrl.contains("wishembed") || embedUrl.contains("embedwish") || embedUrl.contains("streamwish") || embedUrl.contains("strwish") || embedUrl.contains("wish")) {
val docHeaders = headers.newBuilder()
.add("Origin", "https://streamwish.to")
.add("Referer", "https://streamwish.to/")
.build()
StreamWishExtractor(client, docHeaders).videosFromUrl(url, videoNameGen = { "StreamWish:$it" }).also(videoList::addAll)
}
if (embedUrl.contains("doodstream") || embedUrl.contains("dood.")) {
DoodExtractor(client).videoFromUrl(url, "DoodStream", false)?.let { videoList.add(it) }
}
if (embedUrl.contains("streamlare")) {
StreamlareExtractor(client).videosFromUrl(url).let { videoList.addAll(it) }
}
if (embedUrl.contains("yourupload") || embedUrl.contains("upload")) {
YourUploadExtractor(client).videoFromUrl(url, headers = headers).let { videoList.addAll(it) }
}
if (embedUrl.contains("burstcloud") || embedUrl.contains("burst")) {
BurstCloudExtractor(client).videoFromUrl(url, headers = headers).let { videoList.addAll(it) }
}
if (embedUrl.contains("fastream")) {
FastreamExtractor(client, headers).videosFromUrl(url).also(videoList::addAll)
}
if (embedUrl.contains("upstream")) {
UpstreamExtractor(client).videosFromUrl(url).let { videoList.addAll(it) }
}
if (embedUrl.contains("streamtape") || embedUrl.contains("stp") || embedUrl.contains("stape")) {
StreamTapeExtractor(client).videoFromUrl(url)?.let { videoList.add(it) }
}
if (embedUrl.contains("ahvsh") || embedUrl.contains("streamhide")) {
StreamHideVidExtractor(client).videosFromUrl(url).let { videoList.addAll(it) }
}
if (embedUrl.contains("filelions") || embedUrl.contains("lion")) {
StreamWishExtractor(client, headers).videosFromUrl(url, videoNameGen = { "FileLions:$it" }).also(videoList::addAll)
}
if (embedUrl.contains("mixdrop")) {
MixDropExtractor(client).videosFromUrl(url).let { videoList.addAll(it) }
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you an if lover?

Comment on lines +125 to 194
if (embedUrl.contains("voe")) {
VoeExtractor(client).videoFromUrl(url, prefix = "Voe:")?.let { videoList.add(it) }
}
if ((embedUrl.contains("amazon") || embedUrl.contains("amz")) && !embedUrl.contains("disable")) {
val video = amazonExtractor(baseUrl + url.substringAfter(".."))
if (video.isNotBlank()) {
if (url.contains("&ext=es")) {
videoList.add(Video(video, "AmazonES", video))
} else {
videoList.add(Video(video, "Amazon", video))
}
}
"yourupload" -> {
videoList.addAll(
YourUploadExtractor(client).videoFromUrl(serverUrl, headers = headers),
)
}
if (embedUrl.contains("ok.ru") || embedUrl.contains("okru")) {
runCatching {
OkruExtractor(client).videosFromUrl(url).also(videoList::addAll)
}
}
}

if (embedUrl.contains("filemoon") || embedUrl.contains("moonplayer")) {
val vidHeaders = headers.newBuilder()
.add("Origin", "https://${url.toHttpUrl().host}")
.add("Referer", "https://${url.toHttpUrl().host}/")
.build()
FilemoonExtractor(client).videosFromUrl(url, prefix = "Filemoon:", headers = vidHeaders).also(videoList::addAll)
}
if (embedUrl.contains("uqload") || embedUrl.contains("upload")) {
UqloadExtractor(client).videosFromUrl(url).also(videoList::addAll)
}
if (embedUrl.contains("mp4upload")) {
Mp4uploadExtractor(client).videosFromUrl(url, headers).let { videoList.addAll(it) }
}
if (embedUrl.contains("wishembed") || embedUrl.contains("embedwish") || embedUrl.contains("streamwish") || embedUrl.contains("strwish") || embedUrl.contains("wish")) {
val docHeaders = headers.newBuilder()
.add("Origin", "https://streamwish.to")
.add("Referer", "https://streamwish.to/")
.build()
StreamWishExtractor(client, docHeaders).videosFromUrl(url, videoNameGen = { "StreamWish:$it" }).also(videoList::addAll)
}
if (embedUrl.contains("doodstream") || embedUrl.contains("dood.")) {
DoodExtractor(client).videoFromUrl(url, "DoodStream", false)?.let { videoList.add(it) }
}
if (embedUrl.contains("streamlare")) {
StreamlareExtractor(client).videosFromUrl(url).let { videoList.addAll(it) }
}
if (embedUrl.contains("yourupload") || embedUrl.contains("upload")) {
YourUploadExtractor(client).videoFromUrl(url, headers = headers).let { videoList.addAll(it) }
}
if (embedUrl.contains("burstcloud") || embedUrl.contains("burst")) {
BurstCloudExtractor(client).videoFromUrl(url, headers = headers).let { videoList.addAll(it) }
}
if (embedUrl.contains("fastream")) {
FastreamExtractor(client, headers).videosFromUrl(url).also(videoList::addAll)
}
if (embedUrl.contains("upstream")) {
UpstreamExtractor(client).videosFromUrl(url).let { videoList.addAll(it) }
}
if (embedUrl.contains("streamtape") || embedUrl.contains("stp") || embedUrl.contains("stape")) {
StreamTapeExtractor(client).videoFromUrl(url)?.let { videoList.add(it) }
}
if (embedUrl.contains("ahvsh") || embedUrl.contains("streamhide")) {
StreamHideVidExtractor(client).videosFromUrl(url).let { videoList.addAll(it) }
}
if (embedUrl.contains("filelions") || embedUrl.contains("lion")) {
StreamWishExtractor(client, headers).videosFromUrl(url, videoNameGen = { "FileLions:$it" }).also(videoList::addAll)
}
if (embedUrl.contains("vembed") || embedUrl.contains("guard")) {
VidGuardExtractor(client).videosFromUrl(url).also(videoList::addAll)
}
} catch (_: Exception) { }
return videoList
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ctrl+c ctrl+v go BRRRRRRRRRRRRT

Are these sources really ALL using those hosters and really needing to implement this many extractors? Or you just wanted to implement everything imaginable just to make sure?

Comment on lines +203 to +207
return try {
if (client.newCall(GET(videoURl)).execute().code == 200) videoURl else ""
} catch (e: Exception) {
""
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't the app check for this?

Comment on lines +249 to +250
anime.thumbnail_url = document.select(".thumb img").attr("abs:src")
anime.status = parseStatus(document.select(".btn-block.status").text())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No need to create the variable

override fun animeDetailsParse(document: Document): SAnime = SAnime.create().apply {
    title = document.select("h1.title").text()
    ....
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants