knaxuslearn · build · ship

Roadmaps / Embedded Linux / Phase 3

Phase 3 of 4 · Study weeks 9-12

Safe updates and boot

This phase makes fleet-gw survive a bad update and a power cut, and gives you the numbers to say so. Read the RAUC documentation on slots, bundles and the bootloader interface, and the U-Boot documentation on the environment.

Topics every week

  1. Week 98 h
    • Slot layout: A/B root, shared data partition, boot order, the U-Boot environment (redundant copies), size budgets
    • wic

    Hand inThe layout under your slot budget with a size table, and a boot with both slots. The community layer ships one environment copy: adding a redundant one is about 2 hours, a QEMU kill cannot test it, and skipping it must be stated in ADR-3

  2. Week 108 h
    • RAUC: bundles, signing, verity format, rauc install
    • boot attempts, "mark good" and your health unit

    Hand inA signed bundle installed to the inactive slot and booted, with a health check that marks it good

  3. Week 118 h
    • Power-cut testing: the harness, trigger and random cuts, counting outcomes, the rule-of-three bound per stage
    • a bad update that boots

    Hand in100 trials with a table like the worked reference, and the revert of a build whose gwd exits at start

  4. Week 128 h
    • Choosing an update tool on paper (RAUC, SWUpdate, Mender: model, signing, rollback, operating cost; the 2026 SWUpdate signature-check advisory, CVE-2026-109399, is a case to read)
    • tamper and wrong-key tests

    Hand inCheckpoint 3, built in four steps: the layout; the bundle and health check; the 100 trials; the tamper tests (then the captures)

Practice projects

  1. Build the harness that cuts power at a random delay and records which slot booted. Done when 20 trials run unattended and the log says which stage each cut landed in.
  2. Rotate the signing key. Done when an old-key bundle is rejected after the rotation and a new-key bundle installs, and you list what you must do on a fleet to make this safe.
  3. (Optional for a fresher) Install a bundle of a different image version and read the slot status. Done when you can explain the version fields and what downgrade protection would add.

GitHub repos to study

RepoWhat to readWeek
rauc/raucThe documentation, the bundle format and the bootloader interface10
rauc/meta-raucThe image and bundle classes and the system configuration recipe9, 10
rauc/meta-rauc-communityThe U-Boot boot script and the A+B+recovery example9
sbabic/swupdateThe update description and the handlers, for comparison12
mendersoftware/menderThe client and its update model, for comparison12

Milestone checkpoint

Run the 100 trials, the tamper tests and the revert of the broken build. Done when the Checkpoint 3 row passes. Evidence: the trial log and table, the rule-of-three statement, the rejected-bundle captures, the key-rotation notes, ADR-3 (as in the worked reference), ADR-4 (the update tool and the one you rejected, with your measurements). The CVE policy is ADR-5, in Phase 4.

Check yourself

Answer each question in your head first, then open it to check.

  1. Why must the boot order change last in an update?
    Until the new slot is fully written and verified, a cut must leave the old slot as the one that boots.
  2. What does "0 failures in 100 trials" tell you, and what does it not?
    It bounds the failure rate below about 3% at 95% confidence for the mix of cuts you ran; per stage the bound is looser, and it does not cover storage behaviour a QEMU kill cannot show.
  3. What does a signature on an update bundle prove, and what does it not?
    That the holder of the signing key produced it and nobody changed it; not that its contents are safe, or that the key has not leaked.
  4. What is the "mark good" step for?
    It tells the bootloader the new slot works; if it is never set, the boot attempts run out and the bootloader returns to the old slot.
  5. Name two ways a system can boot and still be broken.
    A service that exits at start or hangs, and a network or data problem that the health check does not test.