These are my notes from life as a data-oriented process engineer, plus updates from the personal projects I build on the side. I’m here to share the learnings I’ve picked up over the years - tools, workflows, and hard-won lessons that might be useful to you too.
Beware: Here be dragons (in the data)
In any R&D or continuous improvement environment, progress depends on knowledge transfer. Often, a lot of the most valuable lessons live in people’s heads: the “we tried that in 2014 …” experiments, the one-off protocols, the messy edge cases and details that did not make it into a formal report. Newer teams can try to survey the literature and internal documentation, but without a guide, it’s hard to know what exists, let alone where it’s stored, what to trust, and under what circumstances the data is valid. ...
Presenting high dimensional data to executives isn’t really about data, it’s about steering the decision. Approve capital, change a process, cull an underperforming product. In the wonderland of our imaginations, there exists a two variable system, where the decision is cut and dry – but this is never the case with real data. Data out in the wild always comes with dozens if not hundreds of variables, from complex KPIs to individual sensor data and if you try to show all of it at once, you can expect to open a portal to confusion in PowerPoint form. ...