How to find identical files. AllDup to identify duplicates
Sometimes in everyday computer activity the task is to find duplicate files. There can be many reasons for this: lack of hard disk space, attempts to reduce the entropy in your files, deal with dumped files. different time photos from the camera and many other necessary cases.
On the web you can find a large number of programs that allow you to search for duplicate files. But why look for some programs if a smart tool for such work is usually always at hand. And this tool is called Total Commander (TC).
In this article, I will show all methods based on Total Commander versions 8.5 , in this version, the search for duplicate files has become very rich in functionality.
!!!Small important digression. What is meant by the word duplicate file? Two files are IDENTICAL only when they exactly match bit by bit. Those. any information in a computer is represented by a sequence of zeros and ones. So, files match only when they completely match the sequence of zeros and ones that these files consist of. All talk about the fact that you can compare two files on some other basis is deeply erroneous.
TC has two essentially different methods for finding duplicate files:
- Synchronize directories;
- Search for duplicates;
Their features and applications are best illustrated by examples.
1. Synchronization of directories.
This method is used when you have two compared folders that have an identical structure. This usually happens in many cases, here are a few of them:
- You regularly backed up your working folder. After some time, you need to find out what files have been added or changed since the archive was created. You unpack the entire archive into a separate folder. The folder structure in it practically coincides with the working one. You compare two folders "original" and "restored from the archive" and easily get a list of all changed, added or deleted files. A couple of simple manipulations - and you delete from the restored folder all duplicate files that are in the working one.
- You are working in a folder on network drive and regularly make a copy to your local drive. Over time your work folder has become quite large and the time spent on full copying has become very large. In order not to copy the entire folder each time, you can first compare with the backup and copy only those files that have been changed or added, and also delete in backup folder files that have been removed from the main.
Once you get the hang of the power of this method, you will be able to come up with thousands of situations where the directory synchronization method will be of great help to you in your work.
So, how does everything work in practice. Let's get started.
Suppose we have a main folder "Working", which contains the files that are being worked with. And there is a folder "Archive", in which lies old copy folders "Working". Our task is to find duplicate files in both folders and remove them from the folder "Archive".
Open TC. Open the compared folders in the right and left panes:
Press menu "Commands" - "Sync directories ..."
The directory comparison window opens. 
Next, we need to set the comparison parameters. Put checkboxes in the options "with subdirectories", "by content", "ignore date"
- "with subdirectories"— files in all subdirectories of the specified folders will be compared;
- "by content"- here is the key option that makes TC compare files BITS!!! Otherwise, files will be compared by name, size, date;
- "ignore date"- this option makes TC show different files without trying to automatic detection directions for future copying;
!!! Files will only be compared with the same names!!! If the files are identical, but they have a different name, then they will not be compared!
We press the button "Compare". Depending on the size of the files, the comparison can take a very long time, don't worry. In the end, the comparison will end and the bottom status line (section 1 in the figure) will display the result: 
If the buttons in the "Show" section (section 2 in the figure) are pressed, then you will see the result of the comparison for each file.
- this button enables the display of files that are in the left panel, but which are not in the right;
- this button enables the display of identical files;
— this button enables the display of different files;
- this button enables the display of files that are in right panel, but which are not in the left;
If you have, initially, all display buttons turned off, then the result of the comparison can only be evaluated by the status bar (section 1 in the figure above), in this case we see that 11 files were compared, of which 8 files are the same, 2 files are different, and there is a file in the left panel that is not in the right panel.
To complete our task, it is necessary to leave the display of only identical (identical) files, so all other display buttons are turned off 
Now we have only identical files left, and we can safely delete them in the folder "Archive". To do this, select all files. The easiest way to do this is by pressing universal combination CTRL+A. Or first select the first line with the mouse, then press the key on the keyboard SHIFT and without releasing it, select the last line with the mouse. As a result, you should get something like this: 
In the final step, we right-click on any line and select the item in the menu that opens. "Delete Left"
TC kindly asks us again about our desire, 
and if we click "YES" then it deletes all marked files in the folder "Archive".
After that, the two folders are automatically compared again. If you do not need a second comparison, then the process can be interrupted by clicking on the button "Abort" or press the key ESC on keyboard. If the re-comparison was not interrupted, and we turned on all the display buttons, then we will see this window 
Everybody. The assigned task has been completed. Everybody identical files found and deleted in folder "Archive".
Educational video on the topic
2.Search for duplicates.
Fundamental difference this method from the directory synchronization method is that TC ignores the names of the compared files. In fact, it compares each file with each, and shows us identical files no matter how they are called ! Such a search is very convenient when you do not know the folder structure or the names of the files being compared. In any case, after searching for duplicates, you will get an exact list of identical files.
Finding duplicates I will show on one practical task, finding duplicates of personal photos. Quite often you dump photos from your digital gadgets. Often the situation gets confused, something is reset many times, something is skipped. How to quickly delete files reset multiple times? Very simple!
Let's get started.
Suppose you always dump all your photos into a folder "A PHOTO" on drive D. After all the resets, the folder looks something like this: 
As you can see, some files are in folders named after the shooting date, some are reset to the root of the folder "_New" and "_New1"
To start searching for duplicates, open the folder in which we will search in any TC panel. In our case, this is the folder "A PHOTO"
Next, press the key combination on the keyboard ALT+F7 or choose from the menu "Commands" - "Search for files"
A window opens standard search TC. string "Search files:" leave empty, then all files will be compared. 
Then go to bookmark "Additionally" and tick the boxes "Search for duplicates:", "by size", "by content" and press "To start searching".
The search can take a VERY long time, do not be afraid of this, as it happens great amount comparisons large volume files. At the same time, the status bar shows the percentage of completion. 
When the search ends, a search results window will open, in which we press the button "Files to Panel"
In the search window and in the panel window, identical files are collected in sections separated by dotted lines 
Each section displays the file name and full path to the file. Names of IDENTICAL files can be completely different!
In this case, it can be seen that the same photograph was recorded THREE times, and twice under the same name ( IMG_4187.JPG) and the third time this photo was recorded under a completely different name ( IMG_4187_13.JPG).
Then it remains to select unnecessary identical files and delete them. This can be done manually by highlighting each file by pressing the key Ins. But it takes a long time and is not effective. There are better and faster ways.
So, our task is to remove duplicate files in folders "_New" and "_New1".
To do this, click on additional keyboard, on right big key [+]
. Usually this key in TC selects files by mask. The same operation can be done through the menu "Select" - "Select group"
Greetings, dear reader! Today I will show you a program that looks for identical files on a computer. The program not only finds copies of files, but also, at the request of the user, immediately deletes them. Very convenient in this regard. And copies of files can accumulate so much that you will not even suspect about it. It's just that they can be located in different folders and even on different disks. You may constantly use some of them, but you may have forgotten about their copies.
For example, they downloaded a picture from the Internet, used it for their own purposes and forgot about it. After some time, you needed this picture, but you are too lazy to search on your computer. It's easier to find it online. Download again and get a duplicate file already existing on your computer.
The same can happen with music files. Downloaded to different folders and think that you have it in a single copy. Many PC users make one mistake. When you hook a file with the left mouse button and drag it to another folder located on another disk, then it is not moved, but copied. And this means that the file remained in the same place, and in new folder on another disk got a copy of it.
It turns out that one file is superfluous and only takes free space in computer memory.
Search for identical files
This program has flexible settings with which we can speed up the search.
Let's say we are looking for only one or two disks. We mark them with checkboxes and press the button " Scan" 
But at the same time, the program will find all the files that have copies. And we do not need this, because we, for example, want to find only images.
Search by file type
In this case, go to the "Files and folders" tab. Check the box for the file format. Images are different formats, but the program offers us only four jpg, jpeg, gif, bmp. These are the most popular image formats that almost every user has.
Others that are not in the list must be registered manually. Click the button "Add" In the window that opens, write desired format images. For example from the program Photoshop-(*.PSD) 
OK! We scan and get a bunch of copies to delete. Stop! But they can be systemic. So we go further.
Scan only the folders you need
Let's choose separate folders for scanning. The program will check only them. At the bottom of the program there is a setting " Searched folders" We mark the item " Only specified folders" With these parameters, the disk in the " Disks" tab can not be selected. Yes, and don't forget to check the box here Include these folders even if the corresponding drive is not selected « 
We scan and get the result. Upon completion of the check, DupKiller will switch to the tab " List»Where all found duplicate files will be shown.
Files, in our case, are pictures, sorted into groups. A group consists of two or more files. They are all the same, as they are copies of each other.
What files to delete?
Click on any of the files in the group and you will see a thumbnail of the image in the preview window. Now just scroll the mouse wheel to move through the list and compare the copies with each other.
All information about the file is visible on the program screen. And even if the image is not displayed in the preview window, we can compare files by name, size and type. The first column, titled Path, shows the location of the file. 
Deleting Duplicate Files
We look at this data and select one file in each group. Now the marked files can be deleted by clicking on the “ Delete" or "" button. You can also use the " Delete" key on the keyboard to delete. 
If there are too many files to delete, then it is better to use automatic deletion files. In this case, you yourself choose from which folder to delete the same files. how it works? Select one file in the group with the right mouse button and click on the "Auto-select" button
In the window that appears in the upper block, paths to folders in which there is a similarity between files will be displayed. The same folders are registered in the lower block, but are not marked with checkmarks. We need to select one of these folders in which the files should be deleted. Click "OK"
Here is another annoyance. Each time one of the copies is deleted, a confirmation window appears. 
Feel free to confirm. Disable this notification by going to the settings " Uninstall"And uncheck the item" Ask for confirmation before deletion«
That's it. I superficially showed you the principle of the DupKiller program.
There is a desire to delve into advanced settings « Search settings" and " Other settings«
And I think she does her job well.
Write in the comments how you like this program and how you clean disk space then unnecessary copies?
This is useful to know:
A modern hard drive is created with an impressive volume free space to store information. However, this does not mean that there is no need to "take care" of the disk, freeing it from garbage, duplicated files.
The presence of duplicate files clogs the disk with unnecessary records of information. When the time comes to increase the free space of "littered" disks, significant time will be spent on detecting duplicates - after all, the volume of your hard drive is very large. Therefore, finding and deleting duplicate files must be done on time. Consider a number of applications that can search for such duplicate files.
Total Commander
Let's consider the popular Total Commander as the first application. His functionality wide, it also provides a tool for finding duplicate files located in different directories on your hard drive. The operation we need is available in the general search module.
The search module is called using hot keys or by traditionally going to the main menu, where you need to select the "File Search" function. A call via hotkeys is made by simultaneously pressing the Alt + F7 combination.
The search module offers whole line tabs of general and advanced functions. However, initially we need to select the drive or directory in which we want to conduct discovery. This is done on the tab. general parameters. Then, go to the "Advanced" tab.
In the search module, open "Disks" and select the "D" drive.
Let's go to the "Advanced" tab.
At the bottom of the selected section is a function for finding duplicate files. We only need to configure this feature by checking the required checkboxes next to the desired options. For example, as shown in the image below. In our case, search actions occur on volume D.
Found wanted duplicate Total Commander is placed in a small window where you can barely see anything.
By clicking the button "Files to the panel", we will display them in the main window. Now, you can freely delete them if you are sure that such an action will not cause unstable work programs and the system as a whole.
We get the result in the main window.
Finding duplicate files in CCleaner
Recently updated CCleaner also provides a function to find duplicate files. This application is a well-known and popular "junk cleaner" on the PC, so the presence of such an option in it is absolutely natural. The "Search for duplicates" function is easy to find on the "Service" tab of the program.
CCleaner detects duplicate files by several parameters: name, size, date, content. It also offers a number useful options, which allow you to skip or add to our operation a number of special files: hidden, system, read-only, zero size. To start the search module, just click the "Find" button.
After performing the operation, the user has the opportunity to select and delete the detected duplicates.
Size Explorer Pro
This application is powerful tool examining disk and attached volumes disk devices. One of its functions is to search for duplicate files.
In order for SizeExplorer to start detecting duplicates, it first needs to analyze the section where it needs to search.
The menu that opens will offer you a number of functions, among which is "Find Duplicates". Clicking this option will open a dialog with search module configuration options.
Let's define the detection functions.
Let's search for duplicate files according to the size-name-content scheme. It will take some time to detect, but after the end of the process in a separate window we will see complete list duplicates. At the same time, the program gave us many duplicates of incomplete similarity, according to a certain percentage of similarity.
Search results for identical files:
This is how you can select duplicates:
Let's use Ainvo Duplicate File Finder
Another program of interest is available on the Internet for detecting duplicate files. This is Ainvo Duplicate File Finder. This application is easy to use, so even the most inexperienced can use it. advanced user computer technology. To use the program, just click the mouse "Search for identical files".
The application immediately starts the search, which lasts quite a long time, so you have to be patient.
First, the disks are examined, then the application processes the found material, and only after that additional processing takes place, which results in the detection of duplicate files. To see the found duplicates, you will need to open the "Identical files" window.
As you can see, two sections are selected in the settings for the operation.
Master Ainvo has completed the operation - now we can open a window with links to duplicates.
As you can see, there are a lot of duplicates and the program immediately highlights files that are repeated, in its opinion.
There are 2137 duplicates selected that can be removed using the wizard. Also, it is possible to view a duplicate report.
The report opens in a web browser.
Summary
The hard drive needs maintenance from time to time. One of the preventive procedures is cleaning the volumes of garbage, a large proportion of which are duplicated files. To detect duplicate files located in different directories, you can use special programs. Among the most worthy applications that can detect duplicate files are: Total Commander, CCleaner, SizeExplorer Pro, Ainvo Duplicate File finder.
Deleting duplicate files (copies of the same file) or folders can free up disk space and reduce unnecessary garbage in the system, which has a positive effect on performance systems. Sometimes duplicates are created by the user himself (for example, the same photos are copied to different folders located inside the same physical disk), in some cases they remain after using different software. The function of searching for unnecessary copies is performed by many applications, most of which are distributed under the free status ( for free). Let us describe the operation of several such applications.
Removing duplicates with Total Commander
Using Dupeguru
This software is designed solely to scan the system for copies and does not perform other functions.
AllDup to identify duplicates
The design of the program is made in a beautiful and easy-to-use style. Immediately after starting, a window appears with a brief leadership to find duplicates. Can search all over physical disk in its entirety by ticking the desired local disks.
Or add separate folders by clicking on the arrow next to the " Source Folders
» and selecting in the drop-down list « Add folders». 
Let's try to find duplicates in the Games folder in local disk"FROM:\" 
Activate the tab Search Method”, set the criteria. By default, the option to search for duplicates only by similar names, if necessary, tick the necessary settings. It is advisable to enable search and extensions, otherwise the software may show files with similar names, even if they are not copies of each other. 
After, press the button search start.
When completed, a new window will open with results, in which we mark junk files , click right click mouse by file, which is supposed to be cleared of duplicates (open context menu) and choose " Remove all other files from this group permanently to delete copies. In this case, the original itself will remain untouched, only duplicates will be deleted.
How to use DuplicateCleaner Free
A very thoughtful application for finding and removing duplicates on a computer. At the same time, it is possible to perform Search all files in normal mode, search for copies of images or music, or identical folders. 
Launching the application. First of all, we set up the search criteria, for this, in the section " Additional options
» mark the setting « With the same name"and go to the tab" scan path» 
Choose a folder, press the button additions the selected folder to the included paths and click the button " Scan». 
We are waiting for the end of the scan, which will display statistics found duplicates in separate window. We close the window. 
In the tab " Duplicate Files» list displayed the same files, mark unnecessary ones and press the menu " View", select the item" File deletion» 
In the window that opens, click " Delete file(s)". In this case, it is desirable to include deletion in shopping cart so that you can recover the files you need.
