Roadmaps / C++ / The milestone project
Read first
The milestone project
Scenario. A trading firm's research desk replays recorded exchange feeds to test strategies. bookd reads a binary feed of order messages (add, modify, cancel, execute), keeps an order book per symbol and answers "best bid and offer" queries. Peak load is 1,000,000 messages a second across 500 symbols. Targets: p99 processing time at or under 5 microseconds per message on one core; no heap allocation on the hot path after warm-up; clean runs under AddressSanitizer, UndefinedBehaviorSanitizer and ThreadSanitizer; a parser that survives 10 minutes of fuzzing; a C API that other languages can load.
| Phase | Checkpoint | Done when |
|---|---|---|
| 1 | Correct core | The parser and one order book pass 50 unit tests under ASan and UBSan. A catalogue of six undefined behaviours has a test and the compiler output showing what the optimiser did. A seeded use-after-free (a reference into a vector that grows) and a seeded signed overflow are each caught by a sanitizer in CI. A counter proves zero heap allocations on the hot path. |
| 2 | Concurrent | A parser thread feeds a book thread through a single-producer single-consumer queue. ThreadSanitizer reports no race over 1,000 runs. A seeded race (a plain counter shared between threads) is caught. A stress test moves 100 million messages with none lost, duplicated or reordered. On a weak-memory machine or model, the queue still passes (Senior and above). |
| 3 | Fast | A benchmark with spread reports p50, p99 and p99.9 per message at 1,000,000 messages a second. The book layout is chosen from a three-way comparison. perf shows cache and branch counters before and after the change, and p99 meets the target. A seeded slowdown (a hidden allocation in a log line) is found from a flame graph. |
| 4 | Shipped | CMake presets build with GCC and Clang and run tests in CI. libFuzzer runs the parser for 10 minutes with no crash, and a seeded crash is found within 60 s. A C header exposes bookd and an ABI check fails when a struct changes. The build uses hardening flags and produces an SBOM. A crash produces a report with symbols. |
| W | Work version (Staff and Principal) | On a real codebase: add one sanitizer or fuzz target to CI, fix what it finds and record the defect rate. Pass: a target agreed in writing beforehand is met, or the number of sanitizer findings falls at least 5 times over the same code. The ADR is signed by a named stakeholder, scored by someone other than you, and a recheck is booked three months later. Timebox: six weeks alongside Phases 3 and 4. |
Evidence to hand in. A repository whose README opens with the problem and the numbers above; one ADR (architecture decision record) per decision, each naming a rejected option; raw measurement output and a test report; a one-page design doc; a diagram of the message path. Skeletons: Checkpoint 1 is a table of six undefined behaviours (code, what the standard says, what the compiler did, the sanitizer that catches it). Checkpoint 3 is the layout comparison below with your own numbers and a perf summary. Checkpoint 4 is the fuzz log, the ABI check output and the SBOM.
| Level | What you build and hand in |
|---|---|
| Fresher | Core scope: one symbol, a mutex queue, the four checkpoints, ADRs 1 to 4. |
| Senior | Adds the lock-free queue, a weak-memory run, a benchmark in CI and ADRs 5 to 7. |
| Staff | Adds written decisions with numbers: a build and ABI policy, a sanitizer rollout plan, and the cost of the hardware the targets need. |
| Principal | Adds a memory-safety strategy for a codebase with a measured outcome from the work version, and a checklist other teams reuse. |
Do this at work. A solo lab is not promotion evidence. Take a real C++ codebase and run Checkpoint W on it. Start with a baseline: the sanitizer findings and the crash rate over a month. Make one change. Measure again. A promotion committee accepts the real decision and who made it, the named stakeholder who relied on it, before and after numbers, and a result checked again three months later.
Solo version. With no real codebase, use a named open-source C++ project, build it with sanitizers and write a one-page review of the findings. Ask a peer or mentor to check your measurements. Replay a recorded feed file instead of live traffic. The reference feed generator is part of the checkpoint 1 work.
Laptop scale. Everything runs on a laptop. Absolute numbers depend on your CPU, so report your machine's model and frequency settings, and compare layouts on the same machine. No cloud spend is needed.
Honest sizing. A working order book takes an experienced engineer a day or two. The checkpoints add sanitizers, a lock-free queue verified under a race detector, benchmarks with spread and a fuzzer, which is where the hours go. Phase 2 is the heaviest, so use the extra fresher week there.
Worked reference: what a "3 Proven" hand-in for Checkpoint 3 looks like
The numbers are illustrative. The shape (a layout that matches the access pattern beats a tree on cache misses) is what you should expect. Hand in your own measurements.
| 500 symbols, 1,000,000 messages a second | std::map per price | Sorted std::vector of levels | Flat array indexed by price tick |
|---|---|---|---|
| Heap allocations per message | 0.4 | 0.02 | 0 |
| L1 cache misses per message | 11 | 3 | 1.5 |
| p50 | 1.4 us | 0.6 us | 0.25 us |
| p99 | 9.0 us | 3.1 us | 1.2 us |
ADR-3 excerpt: Index price levels in a flat array by tick. Prices in the recorded feed move inside a band of 4,000 ticks around the last trade, so an array of 4,000 levels (8 bytes each, 32 KB) fits in L1. A tree touches a new cache line per level of depth. Rejected: a sorted vector, because inserts in the middle move memory and the p99 grows with the band. Rejected: a hash map, because we need the best price in order. Revisit if the feed has instruments whose prices jump outside the band; the fallback is an overflow tree.