Standalone mode

[petrkr]

Looking for something tool like this, but standalone.

Use-case:

I have lot of recovered photos from faulty SD cards. Some files are littlebit corrupted by fragmentation, so is impossible to find duplicates by simple MD5SUM.

Because some of those data was already backed in photo library, so there may exist original not corrupted file.

But I do not want upload those backups to immich, but I want just point to set of folders like

• /mnt/backup/SD1 (files from testdisc)
• /mnt/backup/SD2
• /mnt/data/cloud/Photos (mostly original files before SD crashed)

and it will find duplicates over all files.

Now I have immich pointed to /mnt/data/cloud/Photos as read-only External library from Nextcloud, but this tools want immich upload dir (which is empty, since I using external one)

Regards Petr

[eddified]

Have you tried Krokiet? https://github.com/qarmin/czkawka/blob/master/krokiet/README.md

I have used Krokiet and it's predecessor Czkawka in the way you describe: Point at folders and let it do similarity hashing on all files. It has been a very good tool for me.

[petrkr]

Czkawka

I can try. Seems its new project. I solved this actually like 10 years ago and now I remember about that when I saw this deduper which uses AI/ML for check similarity. But I did not had idea to trying to look for something already done

[eddified]

I really like Czkawka. It uses similarity hashing -- it does not use AI/ML as far as I know (to be honest, I don't know much about similarity hashing). But that doesn't mean it's not good at what it does.

Czkawka is not a new project, depending on your definition of "new" I guess. Czkawka is no longer being developed -- the developer has moved the functionality to a new project called "Krokiet" that is basically the same thing. But I still like Czkawka because last I checked, Krokiet did not have a feature that I really liked: after grouping images into similar groups, you can use a regex to select all images from a certain folder. Example: if you want to detect duplicates and then remove all the duplicates on /mnt/backup/SD2, then you enter a regex that matches "/mnt/backup/SD2", and then easily delete them all at once. Krokiet has mostly similar functionality, but last I checked did not have "select based on regex" feature, which Czkawka has.