Michael — thank you for reading thoughtfully, for the kind words and for a good point. You're right on the general principle, and it deserves more care than the sentence gave it.
The precision of an effect estimate is not bounded by the precision of the underlying level. NASA-GISS puts the uncertainty of global mean temperature at 0.5 degrees C, and in fact shifted its official estimate in 2000 from 15 degrees to 14 degrees. But the annual change has an uncertainty of only 0.06 degrees C. The level shift in 2000 didn't modify any of the annual change estimates.
Your handkerchief case is the limiting version: pink versus green is zero plus or minus zero, tighter than any population figure will ever be. So "we don't know world deaths that precisely" is not, by itself, a reason the effect of USAID can't be known that precisely. Point taken.
But I don't think the objection applies to the discussion in my book.
Your example gets its precision from a mechanism argument, not from estimation. Nothing was measured, so nothing propagates. It shows that some effects are known better than aggregates — it doesn't get this one there.
Also, over a billion people died worldwide from 2001 to 2021. The claimed interval is roughly plus or minus 6.3 million, around 0.5%, and the point estimate is 116% of the entire measured decline in global mortality over the period. This isn't a small difference between two large numbers with errors that cancel. It's a re-attribution of nearly the whole aggregate, computed from the aggregate. An estimate can't carry less uncertainty than its inputs when the estimate is most of the input, and UN all-cause mortality for the countries doing the work here isn't known to anything like half a percent.
Finally, the interval is sampling error conditional on the model being right — which is a lot to condition on when nine of forty-eight candidate controls were chosen after the fact and nothing was pre-registered. It's arithmetic inside the model, not a statement about how well we know lives saved.
Separately from all of that: eight significant figures on a counterfactual body count is a bit much no matter how the interval is derived.
Michael — thank you for reading thoughtfully, for the kind words and for a good point. You're right on the general principle, and it deserves more care than the sentence gave it.
The precision of an effect estimate is not bounded by the precision of the underlying level. NASA-GISS puts the uncertainty of global mean temperature at 0.5 degrees C, and in fact shifted its official estimate in 2000 from 15 degrees to 14 degrees. But the annual change has an uncertainty of only 0.06 degrees C. The level shift in 2000 didn't modify any of the annual change estimates.
Your handkerchief case is the limiting version: pink versus green is zero plus or minus zero, tighter than any population figure will ever be. So "we don't know world deaths that precisely" is not, by itself, a reason the effect of USAID can't be known that precisely. Point taken.
But I don't think the objection applies to the discussion in my book.
Your example gets its precision from a mechanism argument, not from estimation. Nothing was measured, so nothing propagates. It shows that some effects are known better than aggregates — it doesn't get this one there.
Also, over a billion people died worldwide from 2001 to 2021. The claimed interval is roughly plus or minus 6.3 million, around 0.5%, and the point estimate is 116% of the entire measured decline in global mortality over the period. This isn't a small difference between two large numbers with errors that cancel. It's a re-attribution of nearly the whole aggregate, computed from the aggregate. An estimate can't carry less uncertainty than its inputs when the estimate is most of the input, and UN all-cause mortality for the countries doing the work here isn't known to anything like half a percent.
Finally, the interval is sampling error conditional on the model being right — which is a lot to condition on when nine of forty-eight candidate controls were chosen after the fact and nothing was pre-registered. It's arithmetic inside the model, not a statement about how well we know lives saved.
Separately from all of that: eight significant figures on a counterfactual body count is a bit much no matter how the interval is derived.