Skip to content

Commit

Permalink
Bug fix
Browse files Browse the repository at this point in the history
Bootstrap sites use a trailing slash. I remove this for internal logic and this is what is saved. But when comparing new scenes to what has been search the trailing slash caused this logic to never go true.
  • Loading branch information
pops64 committed Nov 12, 2024
1 parent 6d93807 commit 48b8aea
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion pkg/scrape/badoinkv2.go
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ func BadoinkSitev2(wg *models.ScrapeWG, updateSite bool, knownScenes []string, o
})

siteCollector.OnHTML(`div.video-grid-view a`, func(e *colly.HTMLElement) {
sceneURL := e.Request.AbsoluteURL(e.Attr("href"))
sceneURL, _ := strings.CutSuffix(e.Request.AbsoluteURL(e.Attr("href")), "/")
// If scene exist in database, there's no need to scrape
if !funk.ContainsString(knownScenes, sceneURL) {
sceneCollector.Visit(sceneURL)
Expand Down

0 comments on commit 48b8aea

Please sign in to comment.