Expand description
Bounded writer-thread pool that decouples disk writes from rayon CPU workers (issue #569, phase 1).
Rayon worker threads that call fs::write directly stall a CPU
slot for the duration of the syscall. IoPool moves those
writes onto a small dedicated pool of writer threads (2–4) fed by
a bounded std::sync::mpsc channel: producers enqueue
{path, bytes} jobs with IoPool::write and the bounded
channel provides natural backpressure (a full queue blocks the
sender instead of buffering unbounded memory).
§Design constraints
- std-only, tokio-free — per
ADR-0001,
ssgruns one scheduler (rayon) plus plain OS threads; no async executor is introduced here. io_uringis out of scope — that is phase 2 of issue #569 (v0.0.48+, Linux-only feature flag). This module is the thread-pool backend only.- No silent data loss — every write error is captured and
surfaced by
IoPool::flush. Dropping the pool without a finalflush()still drains and joins the writers; any errors that were never observed viaflush()are logged aterrorlevel fromDrop.
§Flush semantics
IoPool::flush is a barrier, not a shutdown: it blocks until
every job enqueued so far has been fully processed (written or
failed), then reports the first captured error (logging any
additional ones). The pool remains usable afterwards, so a build
phase can flush() between batches and reuse the same threads.
§Examples
use ssg::io_pool::IoPool;
use tempfile::tempdir;
let dir = tempdir().unwrap();
let pool = IoPool::new();
pool.write(dir.path().join("a.html"), b"<p>a</p>".to_vec()).unwrap();
pool.write(dir.path().join("b.html"), b"<p>b</p>".to_vec()).unwrap();
pool.flush().unwrap(); // barrier: both files are durably on disk
assert_eq!(std::fs::read(dir.path().join("a.html")).unwrap(), b"<p>a</p>");Structs§
- IoPool
- A small pool of dedicated writer threads fed by a bounded channel.