# Duplicate finder that acts safely

> Finds byte-identical files across one folder, both panels or any set of folders (SFTP too), pre-selects every copy but one, and deletes to the Recycle Bin.

- Canonical: https://cwdio.com/features/duplicate-finder/
- Updated: 2026-08-16
- Product: Cwdio File Manager (cwdio) — https://cwdio.com

The duplicate finder answers "do I have this twice?" and then lets you act without risk: it groups files
that are genuinely identical, pre-selects every copy but one, and sends the rest to the Recycle Bin — or,
on the same drive, replaces them with hard links so every path keeps working while the space comes back.

## What it does

- **Byte-identical, efficiently.** Group by size, hash the first 4 KiB of size collisions, fully hash only
  the survivors. A multi-gigabyte file that differs early is settled without being read to the end.
- **One or many roots.** The active folder, both panels, or any set of folders picked with *Add folder…*;
  roots can mix local and `sftp://`. Progress (*Scanning… N files hashed*) is live and the scan cancels on
  close.
- **A keeper per group.** Shortest path · Newest · Oldest, applied to all groups at once and remembered.
  Groups are listed most-reclaimable first with the total space to gain.
- **Focus the reclaim.** Hide groups under a minimum size (≥ 1, 10 or 100 MB) or not matching a name mask
  (`*.jpg;*.png`); a hidden group is never touched by an action.
- **Delete to the Recycle Bin.** Checked copies go through the standard delete confirmation to the trash;
  a group that drops below two copies disappears from the list.
- **Replace with hard links.** Same-volume only, best-effort, two-step confirm; undoable — <kbd>Ctrl+Z</kbd>
  copies the keeper's bytes back into each linked path.
- **Reveal.** Any file jumps the panel to it.

## How to use it

1. Navigate to the folder — Downloads, a photo dump, an old backup — and run Tools ▸ Find duplicate files…
   (or the palette). For "do I already have these somewhere else?", put the other place in the second
   panel and use *Find duplicates across both panels…*.
2. Pick the keep strategy that matches your intent: Newest for downloads, Shortest path for a tidy library.
3. Set a minimum size if you only care about the big wins.
4. Delete — the files are in the Recycle Bin if you change your mind — or hard-link if you want the paths
   to keep working.

## Under the hood

The scan runs in the host process; only the grouped report crosses to the window. Over a WSL distribution
the file collection runs natively in the distro (one `find`), then hashes through the ordinary provider.
The keeper choice is deterministic: a known date beats an unknown one, then the shallowest path.

## What it deliberately does not do

- It does not group by name, date or "looks similar". Image-similarity matching is not part of it — two
  photos are duplicates only if their bytes are.
- Hard-linking never touches the keeper and never crosses volumes; cross-volume, already-linked and remote
  pairs are skipped and reported, not errored.
- It never deletes permanently. The Recycle Bin is the only destination.

## Related

- [Undo and the safety net](/features/undo-and-safety-net/) — the trash and the undo the finder relies on.
- [Size map](/features/size-map/) — when the problem is size, not repetition.
- [Compare and sync](/features/compare-and-sync/) — same-named files that differ, rather than different names that match.

## Frequently asked questions

**How does it decide two files are duplicates?** By bytes, not names. Files are grouped by size first — a unique size cannot have a duplicate, so most files are never read — then same-size files are hashed over their first 4 KB, and only those that also share that head are fully hashed to confirm. Files of 4 KB or less are settled by the head alone. Zero-byte files and symlinks are skipped.

**Which copy does it keep?** One per group, chosen by the keep strategy: Shortest path (the default), Newest or Oldest, applied to every group at once and remembered. The kept file is starred; the others are pre-checked for deletion. You can change any checkbox before acting.

**What are “replace with hard links”?** Instead of deleting the checked copies, each becomes a hard link to its group's keeper: every path still exists and opens, but the bytes are stored once. It works only on the same NTFS volume, is confirmed with a hint about the semantics (editing one changes all), and Ctrl+Z breaks the links back into independent copies.

**Can it search more than one folder?** Yes. Start it on the active folder, on both panels (View ▸ Find duplicates across both panels…), or add folders with the Add folder… button — a local folder and an SFTP one can be compared in the same scan. Overlapping roots do not make a file a duplicate of itself.