Roadmaps / Python / The milestone project
Read first
The milestone project
Scenario. A fleet of 5,000 devices sends JSON events (up to 2 KB) to ingestd. The service validates each event, runs a CPU-heavy transform (parsing and normalising a payload), and writes batches to Postgres. Load is 2,000 events a second on average with a peak of 6,000 for 30 seconds. The node has 2 CPUs and 2 GB. Targets: p99 acknowledgement at or under 200 ms at 2,000 events a second; no event lost when the process is killed after an acknowledgement; RSS at most 400 MB; mypy --strict passes on the core package; the service starts in 3 s or less; a seeded vulnerable dependency fails CI.
| Phase | Checkpoint | Done when |
|---|---|---|
| 1 | Measured interpreter | A single-process baseline handles events and a profile shows where the time goes (top three functions with percentages). A table shows the transform on the standard build, the free-threaded build and a process pool, with predicted and measured throughput. A seeded slowdown (an accidental quadratic string join) is found from the profile and fixed, with before and after numbers. |
| 2 | Typed core | mypy --strict and Ruff pass on the core package. Models validate every event; a property test with Hypothesis throws 10,000 random payloads at the validator and none crash it. A seeded bug (a mutable default argument shared between requests) is caught by a test and you write down which tool would have caught it. |
| 3 | Fast and concurrent | The service runs on asyncio with a bounded queue, a worker pool for the transform and batched writes. At 2,000 events a second the p99 is 200 ms or less over a 5-minute run with spread reported. In 20 runs of kill -9 during a load run, every acknowledged event is in Postgres after restart. A seeded blocking call in the event loop is found from a trace and fixed. |
| 4 | Shipped | CI runs tests, types, lint and pip-audit. A seeded vulnerable dependency fails the build. The image builds from a lockfile, runs as a non-root user and starts in 3 s or less. An upgrade rehearsal moves the service from the previous to the current CPython and reports what changed. A memo says where Python was enough and where you would use another tool. |
| W | Work version (Staff and Principal) | On a real codebase: add one gate (strict typing on one package, or a profile-based performance test) to CI, fix what it finds and record the result. Pass: a target agreed in writing beforehand is met, or the number of findings falls at least 5 times over the same code. The ADR is signed by a named stakeholder, scored by someone other than you, and a recheck is booked three months later. Timebox: six weeks alongside Phases 3 and 4. |
Evidence to hand in. A repository whose README opens with the problem and the numbers above; one ADR (architecture decision record) per decision, each naming a rejected option; raw measurement output and a test report; a one-page design doc; a diagram of the request path. Skeletons: Checkpoint 1 is the three-way table below with your numbers and the profile. Checkpoint 3 is the 20-kill table (kill, acknowledged, present after restart) and the latency report with spread. Checkpoint 4 is the CI output with the failing and passing dependency check.
| Level | What you build and hand in |
|---|---|
| Fresher | Core scope: one service, one worker pool, the four checkpoints, ADRs 1 to 4. |
| Senior | Adds the free-threaded comparison, a typed public API, a benchmark in CI and ADRs 5 to 7. |
| Staff | Adds written decisions with numbers: a typing and packaging policy, an upgrade plan and a cost model. |
| Principal | Adds a decision on a Python standard for several teams, with a measured outcome from the work version and a checklist other teams reuse. |
Do this at work. A solo lab is not promotion evidence. Take a real Python service and run Checkpoint W on it. Start with a baseline over a month: type errors, production exceptions and p99. Make one change. Measure again. A promotion committee accepts the real decision and who made it, the named stakeholder who relied on it, before and after numbers, and a result checked again three months later.
Solo version. With no real service, use a named open-source Python project, add strict typing to one package and write a one-page review of what you found. Ask a peer or mentor to check your measurements. Generate events with a seeded random generator.
Laptop scale. 2,000 events a second runs on a laptop with a Postgres container. If your machine reaches less, report what it reaches, state the CPU model and extrapolate in the memo. Nothing needs cloud spend.
Honest sizing. A working ingestion service takes an experienced engineer a few days. The checkpoints add profiling, a free-threaded comparison, crash tests and an upgrade rehearsal, which is where the hours go. Phase 3 is the heaviest for a fresher (async, pools and crash tests together), so use the extra fresher week there.
Worked reference: what a "3 Proven" hand-in for Checkpoint 1 looks like
The numbers are illustrative. The shape (processes win on CPU-bound work today, free threading is close and costs less memory, and neither helps I/O-bound work) is what you should expect. Hand in your own measurements.
| CPU-bound transform, 1 KB events, 4 workers | Threads, standard build | Process pool, standard build | Threads, free-threaded build |
|---|---|---|---|
| Throughput (events/s) | 2,300 | 8,100 | 7,400 |
| RSS (MB) | 95 | 310 | 140 |
| Startup (s) | 0.4 | 0.9 | 0.5 |
| Third-party packages that worked unchanged | all | all | all but one (needed a rebuild) |
ADR-2 excerpt: Run the transform in a process pool on the standard build. Threads on the standard build gave 2,300 events a second because the global interpreter lock (GIL) lets one thread run Python code at a time. The free-threaded build reached 7,400 with a third of the memory of processes, but one dependency needed a rebuild and the build is still marked experimental for production use by some vendors. The pool meets the 2,000 events a second target with margin, so the extra risk is not worth the 300 MB saved. Rejected: threads on the standard build, which cannot reach the target. Rejected: the free-threaded build, for now. Revisit when all dependencies ship free-threaded wheels, or if the memory limit drops below 300 MB.