Too many Duplicate MP3's!! Help me please :(

[lagrz]

Lately I noticed that my music folder has grown from being 30 gigabytes jumping to well over 80 gigabytes. I noticed that the majority of them are duplicates (possibly because I backed up my iPod). I would consider using the show duplicates option on iTunes but it really is a drag having to individually select each file (even using control click) and deleting them. I found a few tools online that may help me with this issue in an automated way but I personally don't want to dish out $29. Is there any freeware or open-source program that I can use to automatically delete duplicate music files? Keep in mind that I have it all backed up on an external drive and can use any platform from Windows, OSX, or Linux.

Any Help would be appreciated thank you :)

[fishtoprecords]

What is a duplicate? Do you mean exactly the same file?
if so, you can compare md5sum and see which are identical.

But with MP3 files, the same audio can have different tags, and then the checksums are different.

Can you program, say a little Perl? you could right a small perl program to compare the checksums. There are even libraries in CPAN to read the audio portions and the tags, so you could compare them.

I can write the code, but it would take me more than $30 worth of time, so I'd probably just pay.

[zeronome]

i'd like to know as well. i have much many songs, and have neglected cleaning them up.

[lagrz]

it is pretty much same copy of file just name and name1 on the folders. i would do perl but my scripting ability is pretty bad.

[fishtoprecords]

assuming you are on a good OS, you can do it in steps.

shasum * | sort > foo
then look through foo and the dups will be obvious.

[chris3d]

Can you by chance sort the folder by the data associated with the songs?
(artist, title, album, etc)

[john2002]

Duplicate Mp3 Finder (http://www.mp3-duplicate-finder.com)

Pay for it dude. I spent a long time trying to find a free one, then just bit it, and bought this http://www.mp3-duplicate-finder.com

Works awesome. It was under 20 bucks.

Not much good is free anymore.

[lagrz]

I came across clone cleaner lite yesterday and gave it a shot.. (I have everything backed up on my external drive anyway). It's free and seemed to have fit my needs I just re-imported all the music back on iTunes and no longer see the duplicates. Also shaved off nearly 30 gigs which brings it back to what the original folder size was :)

BTW Thx John2002 for your recommendation.

here's the link:
http://www.clonecleaner.com/HCCdownload.html

[fishtoprecords]

Also shaved off nearly 30 gigs which brings it back to what the original folder size was :)


You do realize that having a single folder over 32 gigs is a sin, right?

[lagrz]

its the way iTunes somehow organized it o.O in any case its not all under one folder its all tree'd by artist

[veronicazilla]

I think we're doing this question on the show next week, but since you're asking now...

iTunes:
View > Show Duplicates

Then just delete the dupes and you're good to go.

[lagrz]

Thanks for covering on the show Veronica, just want to point out that I would have considered using that feature but it was too many songs to work with (nearly 30gb of just dupes). In any case I do appreciate the willingness to help :P

[veronicazilla]

Thanks for covering on the show Veronica, just want to point out that I would have considered using that feature but it was too many songs to work with (nearly 30gb of just dupes). In any case I do appreciate the willingness to help :P

Holy crap. I'll have to rethink my answer then!

[tobyweston]

Hmm yeah, i have about 3gb of dupes to clear, and the thought of going through it that way made me procrastinate more :P

I seem to have 4 of some on there which makes it more of a pain :/

[kirillorlov]

You do realize that having a single folder over 32 gigs is a sin, right?

Than I'm very sin full and should be ashamed of my self.

But on the topic of cleaning duplicates, well I'm the kind of guy that sits at his computer doing it manually. Yes it takes a lot of time but you just feel good when you're done and wasted a whole day..PROMISE! :)

[perceman]

If you use a Mac, there's Dupin, which is $15: http://dougscripts.com/itunes/itinfo/dupin.php

[bigmikesndtech]

I just downloaded DUmp3 from http://sourceforge.net/projects/dump3/ and it seems to work pretty well. Finds duplicates based on mp3 fingerprint or sha1. You specify a directory to scan and what file types you want to include, it takes a while to scan but worked pretty well. Its written in java and platform independent but make sure you download the right package for your system.


I think we're doing this question on the show next week, but since you're asking now...

iTunes:
View > Show Duplicates

Then just delete the dupes and you're good to go.

In my iTunes its under the file menu

[murphy1d]

I think we're doing this question on the show next week, but since you're asking now...

iTunes:
View > Show Duplicates

Then just delete the dupes and you're good to go.

BTW - My very cool GOW2 Zune that I won on Tekzilla...had the same problem of dupes. I got a message that the Zune software had to be reinstalled and when I did I had duplcates (57gb) and it took me forevere to remove. Would love to have known about PC dupe-removal software too.

[matthew-syn]

Didn't you guys do a segment on a Plug-in Program for itunes available on a mac and Windows machine that not only made it easier to get the art and lyrics and whatnot for it but it would also find and delete dupes? I wish I could find that episode I want that program lolz!