Phase 3 of 4 · Study weeks 9-12
Concurrency, durability and performance
This phase makes the third checkpoint possible: a store that survives a kill and shows how fast it is.
Topics every week
- Week 98 h
SendandSync- Atomics and memory orderings
- Sharding and lock striping
- Channels against shared state
Hand inA sharded index with a measured crossover against one lock
- Week 108 h
- Model checking with
loom - Stress tests
- The shard handoff
- Deadlock hunting
Hand inA
loommodel of the handoff that fails on a seeded ordering bug - Model checking with
- Week 118 h
- The append log
fsyncand group commit- Recovery
- Crash tests with
kill -9
Hand inThe 20-kill table and the recovery time
- Week 128 h
- Benchmarks with Criterion
- Flame graphs
- Allocation profiles
- Reading a Tokio trace
Hand inCheckpoint 3, with predicted and measured numbers side by side
Practice projects
- Practice 1, the shard count (Week 9): run the index with 1, 2, 4, 8 and 16 shards. Done when: a table shows throughput and p99 for each and you pick the count with a reason.
- Practice 2, the seeded ordering bug (Week 10): weaken one atomic ordering. Done when:
loomfinds the failing interleaving and the strong ordering passes. - Practice 3, the crash (Week 11): kill the process in the middle of a batch, 20 times. Done when: every acknowledged write is present after recovery and the table shows the recovery time.
- Optional, the group-commit window: try 1, 5, 10 and 20 ms. Done when: a chart shows latency against throughput and you say where you would stop.
GitHub repos to study
Existence and last commit were checked on 22 Nov 2026 by shallow clone. Every repository below had a commit in the last 12 months.
| Repo | What to read | Week |
|---|---|---|
| crossbeam-rs/crossbeam | Channels and epoch-based memory reclamation | 9 |
| tokio-rs/loom | How it explores interleavings, and its limits | 10 |
| tokio-rs/tokio | The sync module and the scheduler's work stealing | 12 |
Milestone checkpoint
Build Checkpoint 3. Write your prediction for recovery time and p99 first. Add sharding, group commit and recovery, then run the loom model, the 20 kills and the benchmark. Hand in the kill table, the loom result, the benchmark report, ADR-5 (the commit policy) and ADR-6 (the shard count).
Check yourself
Answer each question in your head first, then open it to check.
What do
SendandSyncmean, and what do they prevent?Sendmeans a value can move to another thread.Syncmeans a shared reference can be used from several threads. They stop non-thread-safe types from crossing threads.Why can
loomfind a bug that a stress test misses?It runs the model through every interleaving it can generate, so a rare ordering is not left to chance.What does
fsyncgive you, and what does it not?It asks the operating system to flush data to the device. It does not help if the device lies about flushing, and the directory entry needs its own sync.Why does group commit raise throughput?
One flush covers many writes, so the cost of the flush is shared.What would make a recovery slower than expected?
A log with no checkpoint to start from, checking every record's checksum without need, and reading the file in small pieces.