Back to Home
Data ScienceJuly 28, 20266 min read

Run the Diagnostics Before You Run the Headline

Run the Diagnostics Before You Run the Headline

Most analytical mistakes I see are not modeling mistakes. They are reporting mistakes: a coefficient gets quoted before anyone checks whether the assumptions behind it survive contact with the data.

In my Formula One study, the first fit looked acceptable on paper. Then the Shapiro-Wilk test returned a p value of 0.0004, which meant the residuals were not normal and the confidence intervals were not trustworthy. The fix was not a better story. It was a Box-Cox transformation and a rebuilt model.

The habit I recommend: plot fitted values against residuals, run a normality test, check variance inflation factors, then look at leverage. Only after those four checks does a number deserve to leave your notebook.

Reproducibility matters just as much. If a colleague cannot rerun your script and land on the same estimate, the estimate is an opinion with decimals attached.