Data & AI
Notebook to pipeline
The analysis that ran once in a notebook is a rumor. Parametrize it, track it, make next quarter free.
The ninth law lands hardest in analysis work, because a notebook is a conversation with a lease: cells run in an order nobody recorded, against data nobody froze, producing numbers everybody forwards. The deliverable can be perfect while the path to it evaporates. The discipline is to make the pipeline the product: parametrized, tracked, runnable from a clean tree, so that next quarter is an argument, not an excavation.
Migration has a simple acceptance test: the new pipeline must reproduce the old answer before it is allowed to produce a new one. Bit-equivalence on the known case is what licenses trust on the unknown one. And the data's discovered quirks, the renamed column, the duplicated batch, are the most expensive knowledge in the room; encode them in the transform, where they compound, not in the transcript, where they die.
How to practice it
Parametrize the report.
This quarter is an argument, not a rewrite. If producing last period's report again costs more than a command, it isn't a pipeline yet.
Reproduce before you improve.
A refactor earns merge by matching the old outputs exactly. Equivalence first, then evolution.
Encode the quirks.
Every data surprise you handle by hand will be handled by hand again. The dedupe, the rename, the unit fix live in tracked code.
Run from a clean tree.
The test of durability is a fresh checkout producing the same report. Anything else is a demo.
Deep scripture: Codeuteronomy X — What is not committed did not happen