knaxuslearn · build · ship

Roadmaps / Data / Setup and lab safety

Read first

Setup and lab safety

You need one laptop with 16 GB of RAM if possible (8 GB works with the smaller scope), 30 GB of free disk, Docker, Python, DuckDB and a container each for Postgres and Kafka. Everything is free. Spark and Flink run in local mode.

RiskRule
Real personal dataUse generated data only. Never load real customer data into a practice lake. The deletion exercise uses fake users.
Backfills overwrite tablesRun backfills on a copy or a branch of the table first. Keep a snapshot you can roll back to.
Cloud storage costsThe core scope needs no cloud account. If you try one, set a budget with a hard cap, and delete the buckets after.
Credentials in pipelinesKeep passwords in environment variables or a secrets file outside Git. Never commit a connection string.