cwdio
Menu
Download

Feature · Organise

Duplicate finder that acts safely

Finds byte-identical files across one folder, both panels or any set of folders (SFTP too), pre-selects every copy but one, and deletes to the Recycle Bin.

Updated 16 August 2026 View as Markdown

cwdio's duplicate finder over a photo library: the first group shows three copies of DSC_0151.jpg at 1.9 MB each — one in a backups folder, one in the photos folder and one named DSC_0151 (1).jpg — with the kept copy starred and unticked and the other two ticked. The footer reads 10 groups, 16 redundant copies, 17 MB reclaimable.
Identical by content, not by name — and one copy per group is starred and left alone before you touch anything. What you delete goes to the Recycle Bin, or becomes a hard link if you would rather keep every path working.

The duplicate finder answers “do I have this twice?” and then lets you act without risk: it groups files that are genuinely identical, pre-selects every copy but one, and sends the rest to the Recycle Bin — or, on the same drive, replaces them with hard links so every path keeps working while the space comes back.

What it does

  • Byte-identical, efficiently. Group by size, hash the first 4 KiB of size collisions, fully hash only the survivors. A multi-gigabyte file that differs early is settled without being read to the end.
  • One or many roots. The active folder, both panels, or any set of folders picked with Add folder…; roots can mix local and sftp://. Progress (Scanning… N files hashed) is live and the scan cancels on close.
  • A keeper per group. Shortest path · Newest · Oldest, applied to all groups at once and remembered. Groups are listed most-reclaimable first with the total space to gain.
  • Focus the reclaim. Hide groups under a minimum size (≥ 1, 10 or 100 MB) or not matching a name mask (*.jpg;*.png); a hidden group is never touched by an action.
  • Delete to the Recycle Bin. Checked copies go through the standard delete confirmation to the trash; a group that drops below two copies disappears from the list.
  • Replace with hard links. Same-volume only, best-effort, two-step confirm; undoable — Ctrl+Z copies the keeper’s bytes back into each linked path.
  • Reveal. Any file jumps the panel to it.

How to use it

  1. Navigate to the folder — Downloads, a photo dump, an old backup — and run Tools ▸ Find duplicate files… (or the palette). For “do I already have these somewhere else?”, put the other place in the second panel and use Find duplicates across both panels….
  2. Pick the keep strategy that matches your intent: Newest for downloads, Shortest path for a tidy library.
  3. Set a minimum size if you only care about the big wins.
  4. Delete — the files are in the Recycle Bin if you change your mind — or hard-link if you want the paths to keep working.

Under the hood

The scan runs in the host process; only the grouped report crosses to the window. Over a WSL distribution the file collection runs natively in the distro (one find), then hashes through the ordinary provider. The keeper choice is deterministic: a known date beats an unknown one, then the shallowest path.

What it deliberately does not do

  • It does not group by name, date or “looks similar”. Image-similarity matching is not part of it — two photos are duplicates only if their bytes are.
  • Hard-linking never touches the keeper and never crosses volumes; cross-volume, already-linked and remote pairs are skipped and reported, not errored.
  • It never deletes permanently. The Recycle Bin is the only destination.

Questions

How does it decide two files are duplicates?

By bytes, not names. Files are grouped by size first — a unique size cannot have a duplicate, so most files are never read — then same-size files are hashed over their first 4 KB, and only those that also share that head are fully hashed to confirm. Files of 4 KB or less are settled by the head alone. Zero-byte files and symlinks are skipped.

Which copy does it keep?

One per group, chosen by the keep strategy: Shortest path (the default), Newest or Oldest, applied to every group at once and remembered. The kept file is starred; the others are pre-checked for deletion. You can change any checkbox before acting.

What are “replace with hard links”?

Instead of deleting the checked copies, each becomes a hard link to its group's keeper: every path still exists and opens, but the bytes are stored once. It works only on the same NTFS volume, is confirmed with a hint about the semantics (editing one changes all), and Ctrl+Z breaks the links back into independent copies.

Can it search more than one folder?

Yes. Start it on the active folder, on both panels (View ▸ Find duplicates across both panels…), or add folders with the Add folder… button — a local folder and an SFTP one can be compared in the same scan. Overlapping roots do not make a file a duplicate of itself.