It is currently Thu Mar 28, 2024 11:16 am

All times are UTC - 7 hours [ DST ]

Recent News:



Post new topic Reply to topic  [ 10 posts ] 
Author Message
 Post subject: Duplicate File Finder
PostPosted: Mon Feb 09, 2009 8:45 pm 
Offline
1.5TB storage
1.5TB storage

Joined: Thu Aug 28, 2008 7:01 pm
Posts: 60
Thanks: 9
Thanked: 2 times in 2 posts
Does anyone have a good duplicate file finder that they use? I think half of my MSS is filled with duplicate files and I would love a good piece of software that would find all of these for me.


Top
 Profile  
Thanks  

Attention Guest: Remove this ad by Registering with the MediaSmartServer.net Forums. It's Free!
PostPosted: Tue Feb 10, 2009 9:03 am 
Offline
Moderator
Moderator
User avatar

Joined: Wed Jul 16, 2008 4:23 am
Posts: 3838
Location: Long Island, New York
Thanks: 94
Thanked: 243 times in 236 posts
windycitycouple wrote:
Does anyone have a good duplicate file finder that they use? I think half of my MSS is filled with duplicate files and I would love a good piece of software that would find all of these for me.


Did you try either of these add ins:

http://forum.wegotserved.com/index.php? ... &showcat=9

_________________
Cheers,
Damian

DIY WHS - 12.5TB and growing, too many gadgets to list

Did you read the MediaSmartServer blog today?
Blog - http://www.adigitalhomeblog.com
"Like a midget at a urinal, stay on your toes!" - Lieutenant Frank Drebin


Top
 Profile  
Thanks  
PostPosted: Tue Feb 10, 2009 11:01 am 
Offline
Top Contributor
Top Contributor
User avatar

Joined: Tue Apr 01, 2008 5:33 pm
Posts: 902
Location: Florida
Thanks: 53
Thanked: 35 times in 35 posts
dbone1026 wrote:
windycitycouple wrote:
Does anyone have a good duplicate file finder that they use? I think half of my MSS is filled with duplicate files and I would love a good piece of software that would find all of these for me.


Did you try either of these add ins:

http://forum.wegotserved.com/index.php? ... &showcat=9


On the link provide by dbone, The Duplication Info Add-In could be better named. It tells you which drives contain the folders where you have turned duplication ON. It will not tell you where duplicated files are like you are seeking. The other Add-In suggested by dbone, sounds like it will do what you want, but I do not have that one installed so I cannot confirm.

_________________
It is the mark of an educated mind to be able to entertain a thought without accepting it.
Aristotle - Greek critic, philosopher, physicist, & zoologist (384 BC - 322 BC)


Top
 Profile  
Thanks  
PostPosted: Tue Feb 10, 2009 6:04 pm 
Offline
1.5TB storage
1.5TB storage

Joined: Thu Aug 28, 2008 7:01 pm
Posts: 60
Thanks: 9
Thanked: 2 times in 2 posts
Yea I saw those, but as the previous poster said only one is a real file finder and the reviews are pretty bad. Sounds like it just compares filename verus actually looking into the file (among other things).

Anyone else?


Top
 Profile  
Thanks  
PostPosted: Tue Feb 10, 2009 6:48 pm 
Offline
Top Contributor
Top Contributor
User avatar

Joined: Tue Apr 01, 2008 5:33 pm
Posts: 902
Location: Florida
Thanks: 53
Thanked: 35 times in 35 posts
windycitycouple wrote:
Yea I saw those, but as the previous poster said only one is a real file finder and the reviews are pretty bad. Sounds like it just compares filename verus actually looking into the file (among other things).

Anyone else?


To be perfectly frank, IMO you are lucky that someone has even developed and add-in to address your problem at this stage of WHS’s life cycle.

I read the review on DupeCleaner and the reviewer did not say it was pretty bad. It does what you want to do according to the review. It is a very early beta so it needs some speed tweaking apparently. If you use it on a few Share Folders at a time it seems it would do the job for you.

I don't know Brent (the developer) personally but I have used another one of his add-ins that is in beta. He actively seeks beta testers for his add-ins. He is responsive to request for improvements and he does not charge you a penny for the software or the support. He is busy because he has a lot of irons in the fire so he might not get your feature request done right away. I’m thinking you might want to try the add-in first and then if you are unhappy with it, you could talk to the developer via his forums or blog. Based on my forum and blog communication experience with Brent I believe he will try to provide any improvements you might suggest. Please keep in mind, if you decide to go this route, that you are asking someone to invest their time and talent for free.

I gather from you post, you might not be happy because DupeCleaner does not look into the file to see if it is a duplicate file (I assume you mean like Drive Extender does to see if a copy of a file is already on the server before it makes a second backup of the same file). If that is the case, maybe you should consider writing an add-in that would meet your needs. Short of writing your own add-in, I don’t think it is likely you will find another product written for WHS, particularly one that is free, that meets your needs as well as DupeCleaner. I suspect it would be fairly difficult to improve the add-in Brent wrote to “look at the first 20,000 bits of a file to see if it is a duplicate file (like DE does). However, I am not and add-in developer, so maybe Brent would try to accommodate your needs if you just asked politely.

_________________
It is the mark of an educated mind to be able to entertain a thought without accepting it.
Aristotle - Greek critic, philosopher, physicist, & zoologist (384 BC - 322 BC)


Top
 Profile  
Thanks  
 Post subject: Duplicate Files
PostPosted: Wed Feb 11, 2009 7:35 am 
Offline
1TB storage
1TB storage
User avatar

Joined: Tue Feb 03, 2009 2:19 pm
Posts: 27
Thanks: 5
Thanked: 0 time in 0 post
Looked around, but thought I'd get a better idea here for my new 487

I've copied everything, including old archive/backups and wanted to find a utility that could search and find duplicate files. A lot of the freeware stuff just seemed lacking.

Any suggestions?

I'm wanting to prune what I have and delete true duplicates.

Thanks

Russ

_________________
EX487; 4.5TB


Top
 Profile  
Thanks  
PostPosted: Wed Feb 11, 2009 6:20 pm 
Offline
1.5TB storage
1.5TB storage

Joined: Thu Aug 28, 2008 7:01 pm
Posts: 60
Thanks: 9
Thanked: 2 times in 2 posts
I didnt ask about one that ran with WHS.

SeaRay33 wrote:
windycitycouple wrote:
Yea I saw those, but as the previous poster said only one is a real file finder and the reviews are pretty bad. Sounds like it just compares filename verus actually looking into the file (among other things).

Anyone else?


To be perfectly frank, IMO you are lucky that someone has even developed and add-in to address your problem at this stage of WHS’s life cycle.

I read the review on DupeCleaner and the reviewer did not say it was pretty bad. It does what you want to do according to the review. It is a very early beta so it needs some speed tweaking apparently. If you use it on a few Share Folders at a time it seems it would do the job for you.

I don't know Brent (the developer) personally but I have used another one of his add-ins that is in beta. He actively seeks beta testers for his add-ins. He is responsive to request for improvements and he does not charge you a penny for the software or the support. He is busy because he has a lot of irons in the fire so he might not get your feature request done right away. I’m thinking you might want to try the add-in first and then if you are unhappy with it, you could talk to the developer via his forums or blog. Based on my forum and blog communication experience with Brent I believe he will try to provide any improvements you might suggest. Please keep in mind, if you decide to go this route, that you are asking someone to invest their time and talent for free.

I gather from you post, you might not be happy because DupeCleaner does not look into the file to see if it is a duplicate file (I assume you mean like Drive Extender does to see if a copy of a file is already on the server before it makes a second backup of the same file). If that is the case, maybe you should consider writing an add-in that would meet your needs. Short of writing your own add-in, I don’t think it is likely you will find another product written for WHS, particularly one that is free, that meets your needs as well as DupeCleaner. I suspect it would be fairly difficult to improve the add-in Brent wrote to “look at the first 20,000 bits of a file to see if it is a duplicate file (like DE does). However, I am not and add-in developer, so maybe Brent would try to accommodate your needs if you just asked politely.


Top
 Profile  
Thanks  
PostPosted: Wed Feb 11, 2009 7:19 pm 
Offline
Top Contributor
Top Contributor
User avatar

Joined: Tue Apr 01, 2008 5:33 pm
Posts: 902
Location: Florida
Thanks: 53
Thanked: 35 times in 35 posts
windycitycouple wrote:
I didnt ask about one that ran with WHS.

Software we talk about on WHS forums like this one usually refer to WHS software.

I'll admit I don't know enought about DE and I think the files where dups are located in the share folders are just formated as NTFS so its possible it might not be an issue, but if you mess with what WHS DE is doing with the files, its certainly possiblie you could trash your WHS system. If you stick with WHS add-ins you should not have that problem. Someone with more knowledge about DE and share folders will have to help you with that.

If you want to deal with the files that are off your WHS before they get on the server, Google is your friend. If you are looking for a reference to someone who has evaluated general (non-WHS based dup cleaners ) then here is a link to start.

Keep in mind that if you are specific about what you want it is easier for others to help you.

_________________
It is the mark of an educated mind to be able to entertain a thought without accepting it.
Aristotle - Greek critic, philosopher, physicist, & zoologist (384 BC - 322 BC)


Top
 Profile  
Thanks  
PostPosted: Fri Feb 13, 2009 8:23 am 
Offline
2.5TB storage
2.5TB storage

Joined: Tue Jan 29, 2008 10:42 pm
Posts: 286
Thanks: 4
Thanked: 13 times in 13 posts
windycitycouple wrote:
Does anyone have a good duplicate file finder that they use? I think half of my MSS is filled with duplicate files and I would love a good piece of software that would find all of these for me.


Although not strictly a duplicate file finder in the strictest sense of the word I use a program called Beyond Compare by Scooter Software to view, compare & synchronise folders - not on the WHS I may add but it does work well across the network.

Obviously this will not be able to look into WHS specific backups etc.

There is a free 30-day trial so you have nothing to lose if you think it might help?

http://www.scootersoftware.com/index.php

I'm still on v2.41 so it's about time I upgraded!

Kind regards,
-=Glyn=-


Top
 Profile  
Thanks  
PostPosted: Mon Aug 22, 2016 9:56 am 
Offline
Max Contributor
Max Contributor
User avatar

Joined: Sat Apr 12, 2008 12:49 pm
Posts: 2226
Location: Casper Wyoming
Thanks: 81
Thanked: 256 times in 251 posts
This is the one I have used for years on my servers. It has a free version, but I have the paid version with free updates for life I believe.
The pro/paid version does a lot more. And you can use it on normal pc's as well.

I run it from a client and point it to a share folder like (Music) and let it do it's thing.
Then I will point it to (Photos) and let it do it's thing.

Not super fast but speed depends on how much stuff you have, but it has lots of customization options.
My scans take about 30 mins on a large music folder.
Then it gives you options to delete all dupes or save in a different folder "Just in case" or delete all dupes except the newest, options like that.

http://www.duplicatecleaner.com/

I tried the free version on an old client just to get the feel of it, it is not a point and shoot, it works better if you custom it a little I found.
Good Luck

P.S. I made an Application Shared folder where I have all My Applications with sub folders like OS and iso and graphics and video and Etc.
I tried to run the dupe cleaner on that folder, no joy, as a lot of the programs use duplicate files and the programs all needs their own version of that file. So could not delete them.
So I had to leave that folder alone.
Music / Photos / Movies / TV Recordings
Maybe personal folders where dupes might be is where I have found most use of it.

P.s.s I paid for the snoopdedupe addin and could not figure it out.

_________________
[X510 CPU/RAM/All 2 TB Red's] [X510 CPU/Sync'ed Backup/All 2 TB Hitachi's]
[X510 /2 in box]
[X710 LIAN LI DIY W2016E] Stablebit Drivepool [EX-503 LIAN LI 5-bay USB3 External]
:twisted: Fear the Cloud :twisted:


Top
 Profile  
Thanks  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 10 posts ] 

All times are UTC - 7 hours [ DST ]


Who is online

Users browsing this forum: No registered users and 19 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB® Forum Software © phpBB Group