Hi All, Due to some awful archiving. I have around 1500 digital still images that have been duplicated triplicated or more in a number of folders,C/D's and DVD's on a number of drives. I would like to know if there is a way I can dump ALL the directories into one folder then run a "find dupes" type program that would ensure that I did not delete any of my original photos. Regards John
I had a script to do this a while back; used find, md5sum and sort, produced a list of all the files where there was more than one with an identical md5sum, and optionally deleted all but one automatically. That was always a bit of a risky option! Also never got around to making it the oldest one, or the one with the most sensible-looking filename or anything. It was pretty random about which one it chose to keep.
I can't find the script now but I could probably hack something up again if nobody knows of any better tools.
On 8 March 2010 20:23, John jaytee@clear.net.nz wrote:
Hi All, Due to some awful archiving. I have around 1500 digital still images that have been duplicated triplicated or more in a number of folders,C/D's and DVD's on a number of drives. I would like to know if there is a way I can dump ALL the directories into one folder then run a "find dupes" type program that would ensure that I did not delete any of my original photos. Regards John
wlug mailing list | wlug@list.waikato.ac.nz Unsubscribe: http://list.waikato.ac.nz/mailman/listinfo/wlug
All the photos were taken on the same camera and the common thread is the date and time although some were resized (for email) the originals could be identified by there size. John
On Mon, 2010-03-08 at 20:33 +1300, Bruce Kingsbury wrote:
I had a script to do this a while back; used find, md5sum and sort, produced a list of all the files where there was more than one with an identical md5sum, and optionally deleted all but one automatically. That was always a bit of a risky option! Also never got around to making it the oldest one, or the one with the most sensible-looking filename or anything. It was pretty random about which one it chose to keep.
I can't find the script now but I could probably hack something up again if nobody knows of any better tools.
All the photos were taken on the same camera and the common thread is the date and time although some were resized (for email) the originals could be identified by there size.
The following threads list a few programs that do the job of finding duplicates (or similar ones): http://ubuntuforums.org/showthread.php?t=570559 http://www.linuxquestions.org/questions/linux-newbie-8/i-need-a-command-that...
Cheers, Peter
doesn't cp have an overwrite-if-larger option?
On Mon, Mar 8, 2010 at 9:31 PM, Peter Reutemann fracpete@waikato.ac.nzwrote:
All the photos were taken on the same camera and the common thread is the date and time although some were resized (for email) the originals could be identified by there size.
The following threads list a few programs that do the job of finding duplicates (or similar ones): http://ubuntuforums.org/showthread.php?t=570559
http://www.linuxquestions.org/questions/linux-newbie-8/i-need-a-command-that...
Cheers, Peter
Peter Reutemann, Dept. of Computer Science, University of Waikato, NZ http://www.cs.waikato.ac.nz/~fracpete/ Ph. +64 (7) 858-5174 _______________________________________________ wlug mailing list | wlug@list.waikato.ac.nz Unsubscribe: http://list.waikato.ac.nz/mailman/listinfo/wlug
Thank you Peter...
The following threads list a few programs that do the job of finding duplicates (or similar ones): http://ubuntuforums.org/showthread.php?t=570559 http://www.linuxquestions.org/questions/linux-newbie-8/i-need-a-command-that...
Cheers, Peter
GQView has a very good duplicate checking and deletion feature and afaik it is installed by default in ubuntu It has options to check via checksum, size, date and also image similarity and can check them in place by simply adding all images into a single collection
Regards Jason
John wrote:
Hi All, Due to some awful archiving. I have around 1500 digital still images that have been duplicated triplicated or more in a number of folders,C/D's and DVD's on a number of drives. I would like to know if there is a way I can dump ALL the directories into one folder then run a "find dupes" type program that would ensure that I did not delete any of my original photos. Regards John
wlug mailing list | wlug@list.waikato.ac.nz Unsubscribe: http://list.waikato.ac.nz/mailman/listinfo/wlug
It was not a default on my system but I have now installed it thanks. Regards John
On Mon, 2010-03-08 at 23:17 +1300, Jason Drake wrote:
GQView has a very good duplicate checking and deletion feature and afaik it is installed by default in ubuntu ___________________________________________
wlug mailing list | wlug@list.waikato.ac.nz Unsubscribe: http://list.waikato.ac.nz/mailman/listinfo/wlug
participants (5)
-
Bruce Kingsbury -
Darryn Brooking -
Jason Drake -
John -
Peter Reutemann