Duplicates

[Baltimorepc]

There was a thread where there was a issue when uploading images it would cause duplicates. Has this been Resolved? if so how can i get rid of all the duplicates with out having to do it manually? As it created alot

[ildyria]

It has been resolved yes. However the duplication removal is a tricky one and may need some DB fiddling
(or wait for a duplicate searcher to be available in Lychee).

Could you check in your DB if a duplicate has the same sha thumbprint ? That would make it easier for me to create such page (rather than just trying to find on names...)

[ildyria]

I was hoping there would be an easy way out. But it seems that it will need either some duplicate finding engineering or creating the duplicate page finding earlier than I expected.

I'll give you a small SQL query to execute too, that should help with checking if there are duplicates. But I need to cook that later tomorrow. First sleep.

[Baltimorepc]

no worries, i appreciate the assistance. Thank you. the only real concern i have about the duplicate naming is when members upload to the same folder they maybe different pics but possibly the same name since they come from different device but same model. But i dont recall seeing any but truthfully i wasent really looking at the names unless i saw the same pic is when i actually glanced to see the name. So its possible there some but i could check if needed.

[ildyria]

We do have a checksum column in the DB which I plan to see if I can use in order to find duplicates (checksum never lie).

[ildyria]

Some useful queries for later...

Photos duplicates (checksum):

SELECT
    a.title as album_title,
    p.id as photo_id,
	p.title as photo_title,
    p.checksum
FROM
    photos as p
JOIN base_albums as a ON p.album_id = a.id
INNER JOIN photos as p2
ON p.checksum = p2.checksum AND p.id <> p2.id
ORDER BY p.checksum ASC;

Photos title duplicate in same album:

SELECT
    a.title as album_title,
    p.id as photo_id,
	p.title as photo_title,
    p.checksum
FROM
    photos as p
JOIN base_albums as a ON p.album_id = a.id
INNER JOIN photos as p2
ON p.title = p2.title AND p.id <> p2.id AND p.album_id = p2.album_id
ORDER BY a.title ASC, p.title ASC;

Pure photo duplicate in same album (title & checksum)

SELECT
    a.title as album_title,
    p.id as photo_id,
	p.title as photo_title,
    p.checksum
FROM
    photos as p
JOIN base_albums as a ON p.album_id = a.id
INNER JOIN photos as p2
ON p.title = p2.title AND p.checksum = p2.checksum AND p.id <> p2.id AND p.album_id = p2.album_id
ORDER BY a.title ASC, p.title ASC;

[ildyria]

That should make it easier for me to track those. :)

[ildyria]

Fix incomming: #2872

[Baltimorepc]

It looks like not only did it delete the duplicates it also added the normal file to that list and deleted them all not just the copies. I thought that list only shown duplicates so i deleted all that shown. and seems it deleted the first one too. so now yea i dont have dublicates but it removed the only copy with it.

[ildyria]

Well it show duplicates AND original. How am I supposed to know which one is the original ? 😅
So yes, if you tick everything, it deletes... everything.

[ildyria]

I thought that was clear.

[ildyria]

@Baltimorepc 57fb224 (#2890) added a warning :)