Funding systems also tend to create the same pitfalls. Just look at the NSF and DARPA project guidelines—"prove quantitative safety improvements." Translated, it means: "Show benchmark results, or you're rejected."

I've gone through a bunch of funding project documents, and everywhere there are requirements like "achieve measurable progress on existing metrics." The problem is: truly innovative safety assessment methods? Those are precisely things that cannot be directly quantified. Systematic incentive mechanisms tend to favor easily measurable aspects, pushing the most valuable research directions into the corners. Isn't this a classic case of metric distortion?