# Bytes stay in the host: a fast Electron file manager

> Why cwdio never routes file contents through Electron IPC, keeps providers and the copy queue in one services process, and publishes a copy only once good.

- Canonical: https://cwdio.com/blog/bytes-stay-in-the-host/
- Updated: 2026-08-15
- Product: Cwdio File Manager (cwdio) — https://cwdio.com

"Electron, so it is slow" is the first objection any Electron file manager meets, and it is a fair one to
raise, because the obvious architecture *is* slow. This is what cwdio does instead, retold from ADR 0015,
the decision that fixed the data path before the remote and archive providers were built on top of it.

## The trap

Electron's inter-process messaging copies its payloads. Structured clone across one boundary costs on the
order of 12 ms per 256 KB and scales linearly with size — so a multi-gigabyte file pushed through IPC is
not slow, it is catastrophic. The `invoke`/`send` channels do not take transferables at all, and the
cross-process `ArrayBuffer` transfer that `postMessage` nominally offers has documented payload-loss and
crash bugs. A file manager whose renderer reads a file and hands the bytes to the main process to write
would pay that cost twice, once per hop, plus main-thread CPU for the privilege.

Two more traps sit next to it. Reading a whole file into memory before writing it uses roughly seventeen
times the memory of a backpressured stream on a large file. And a copy engine that emits a progress event
per chunk fires thousands of IPC messages for one big file — each cheap, together a storm.

## The rule

**Bytes stay in the host.** All filesystem providers — local, SFTP/FTP, archive — live in one **services
process** (an Electron `utilityProcess`), co-located with the provider registry and the operation queue.
When you press <kbd>F5</kbd>, the renderer sends a request the size of a tweet; the queue in the host resolves
the source and target providers and moves bytes disk-to-disk, or socket-to-disk, inside that one process.
What comes back across IPC is control and progress — never content.

The corollaries in ADR 0015 are what make the rule hold under real workloads:

- **Stream, don't buffer.** Providers expose read and write streams; the generic executor pipes them with
  backpressure and an abort signal. Same-provider fast paths are preferred when both ends share a scheme.
- **Coalesce progress in the host.** At most one progress event per ~100 ms and per few megabytes,
  publishing cumulative counters; start, complete, error and cancel are immediate. A multi-gigabyte copy
  produces hundreds of events, not thousands.
- **Move bytes to the renderer only when it truly needs them** — a preview, a hex view — and even then over
  a brokered direct port with chunked, flow-controlled messages, never through the main broker.
- **List by attributes.** Remote and archive listings are built from the directory's own metadata (SFTP
  `readdir` attributes, the ZIP central directory), never a stat per entry over the network.
- **Plugins that provide filesystems load into the same host.** A provider running in the command-plugin
  process would put every read and write back on the wrong side of the boundary.

## What the copy engine does with the freedom

Once the bytes are in one process, the engine can be honest about correctness in ways a
renderer-mediated copy never could.

**Temp, then rename.** A streamed copy writes to a sibling temp file (`<target>.cwdio-copy-<id>.tmp`) and
publishes it with a `rename` only when the stream completes. A failure mid-stream — a dropped SFTP socket, a
wrong archive password discovered mid-decrypt, a corrupt entry — deletes the temp and leaves the target
untouched. Nothing is ever a partial or zero-byte file where a good one used to be.

**Verify** (View ▸ Verify copies, off by default) accumulates a CRC-32 over the source bytes *as they
stream*, reads the temp back, and fails with a typed `INTEGRITY` error before the rename. The queue carries
`verified: true / false / null` all the way to the Operations drawer, and honestly reports `null` when a
target cannot be read back.

**Resume** treats a vanished drive or a dropped connection as `UNAVAILABLE`, not as failure. The operation
moves to a **paused** state, keeps its partial temps, and a watcher polls the endpoints; when they return,
the run continues each partial temp at its byte offset and counts already-copied files as done.

**Per-file skip and retry.** One locked file in a tree records a failure and its siblings carry on; the
drawer shows *N failed*, what is locking each one, and a retry button. A move keeps its entire source when
anything failed.

## The fast paths, and their honest trade-off

The engine streams with a 4 MiB high-water mark rather than Node's 64 KiB default, and files up to 64 MiB
skip the JavaScript stream entirely for the kernel's own copy call — `CopyFileW` on Windows — which was
measured ~1.7× faster than a stream on a single file and picks up SMB copy-offload and ReFS block cloning
that user-space code can never get. Above 64 MiB, and always under verify or resume, the stream is used
because progress and mid-file cancel matter more than the last percent.

One trade-off is stated plainly in the reference: a **fresh** small file inside a **freshly created**
folder copies straight to its target with no temp — a folder that did not exist a moment ago has nothing to
protect, and skipping the second directory write is what took a 20,000-small-files copy from ~985 to ~1,837
files per second. A hard crash mid-copy can therefore leave a partial *new* file — the same as Explorer,
robocopy and `fs.cp` — and a resumed run re-copies it. Overwrites, large files, verified and resumed copies
keep temp-and-rename.

## What it deliberately does not do

- It does not raise the libuv thread-pool size; that measured *worse* for local copies, which are
  write-bandwidth-bound.
- It does not hand-roll a chunked read/write loop below 64 MiB — that loses the kernel offload.
- It does not put file bytes on the renderer's plate to draw a progress bar. The renderer gets counters.

The numbers behind this — listing, copy throughput, traversal — are published on the [benchmarks
page](/benchmarks/) with the environment they were measured on, and regenerated per release. They are an
order-of-magnitude answer to "is Electron slow?", not a boast; the architecture is why they can be that
answer.

## Frequently asked questions

**Does a copy in cwdio go through the browser window?** No. The renderer sends a small request; the copy runs disk-to-disk (or socket-to-disk) inside a separate services process, and only progress messages come back — throttled to a few per second. The bytes never cross Electron's IPC.

**What does 'verified copy' actually check?** A CRC-32 accumulated over the source bytes as they stream, compared against a read-back of the freshly written temp file, before the temp is renamed into place. A mismatch fails the copy with an INTEGRITY error and nothing is published. It is off by default (View ▸ Verify copies).

**Can a copy leave a half-written file?** Streamed copies write to a sibling temp file and publish by rename only when the stream — and, if enabled, the checksum — succeeds; on failure the temp is deleted. The one exception is a fresh small file in a freshly created folder, which copies straight to its target for speed; on failure that partial file is removed and a resumed run re-copies it.