Handling of duplicate files on Coomer creator downloads #314
Replies: 3 comments 8 replies
-
|
Use |
Beta Was this translation helpful? Give feedback.
-
|
Once I noticed this behaviour (because so many early videos were missing from various downloads), I began testing using two urls, each containing two videos, and one video specifically is present in both URLs for a total of four videos to download. Once scraping is finished, it only downloads three videos, so the deduplication appears to be done during scraping. |
Beta Was this translation helpful? Give feedback.
-
|
Not downloading duplicate videos definitely makes sense if I were downloading all files into one directory, but I'm using separate_posts as true in my config file so there are directories being created for each post but without all the post contents inside because one or more items are found in another post. Even if it were a matter of downloading that video once, but copying it to all directories/posts in which it can be found would be useful. |
Beta Was this translation helpful? Give feedback.
-
|
Please attach your logs and the |
Beta Was this translation helpful? Give feedback.
-
|
Looks like there's a problem with coomer downloads, I'll attach both asap as soon as #320 is fixed. |
Beta Was this translation helpful? Give feedback.
-
Beta Was this translation helpful? Give feedback.
-
|
Updates on this issue will be tracked on #337 |
Beta Was this translation helpful? Give feedback.
-
|
Sorry, I tried it on 5.7.2 post4 and using the urls.txt file attached, it still only downloads 3 files instead of 4. |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
|
Works great. Thank you so much! |
Beta Was this translation helpful? Give feedback.
-
Hey, so when downloading a creator's posts from Coomer it starts downloading from the most recent date all the way through until the first post. Because of this behaviour and how CyberDropDownloader handles duplicates, if there's a specific video that the account's posted twice or more, it downloads only the most recent video and not any subsequent videos.
Example:
-Creator has a series of posts going from 2022-2024
-There's a video that the creator has posted once in 2022 and then again in 2024
-I enter the creator's url and download the entirety of their posts
-CyberDropDownloader downloads the video posted in 2024 and doesn't download the one posted in 2022
Ideally I'd like to be able to download both videos and then decide which one I'd like to keep afterwards, or download from the creator's first post up until the most recent post, which will still delete duplicates, but keep the earliest chronological instance of that video (2022 in my example). I've read the wiki and messed with the dupe cleanup settings but haven't found a way to do either, as it seems those settings mostly affect how it deals with duplicates already on the drive at the time of download.
Is there a way to have CyberDropDownloader just download all the content and not do any hash comparisons or avoidance of duplicates OR have CyberDrop download Coomer accounts in reverse order?
Beta Was this translation helpful? Give feedback.
All reactions