Dario Amodei has published a lengthy post arguing that frontier AI development must be paced. There are already plenty of people disagreeing with the proposal he makes, and many are speculating about ulterior motives. Personally, I simply find the argument deeply unconvincing.

Bias alert: I’ve been working in the clinical IT space for over 15 years, including more than a decade in environments where advanced AI systems are developed to assist with clinical diagnosis or, in some cases, make diagnoses directly. I’m entirely sympathetic to arguments around risk management, safety engineering, validation, and objective performance measures.

Read More

Earlier this year, the Leiden Declaration was published, addressing how mathematicians ought to integrate AI tools (primarily LLMs, if we’re being honest). It has since been joined by a letter from Fields Medalists warning of a Severe Misalignment of AI in Mathematics.

Taken together, I think these are some of the most consequential things written about the future knowledge work in the face of AI.

What interests me is that the debate is often framed as being about whether AI is becoming more capable of doing mathematics. I think that misses a deeper question: to what extent does AI progress itself depend on mathematicians? And, further, what role do the current mathematical LLM users contribute to progress?

My thinking here is straightforward. Frontier users are not merely consumers of frontier AI systems: they are increasingly part of the process by which those systems improve. If that relationship weakens, some of the apparent pace of AI progress could weaken with it.

Read More