Anthropic releases AI proficiency report

The company analyzed 9830 conversations with Claude to understand how users interact with the model. The study uses a 4D framework that identifies 24 skills for effective AI collaboration, 11 of which can be measured in chat logs.
The key finding: an iterative approach where users refine answers and develop ideas correlates with higher-quality output. These users are 5.6x more likely to question the AI’s logic and 4x more likely to notice missing context.
However, when generating final artifacts, users become less critical. The likelihood of fact-checking drops by 3.7 percentage points, and critical analysis of arguments falls by 3.1 points. Anthropic advises against taking results at face value and recommends always asking follow-up questions.
Up next
Anthropic co-founder Chris Olah warns on AI at the Vatican
Anthropic’s Chris Olah says AI systems show internal states akin to human emotions. The tech isn’t just code; it mirrors our own cognition. Developers can no longer ignore the ethical implications of their work; this conversation is now happening at the level of global institutions.
Related stories
- Anthropic bought a biotech with one full-time employee
- A perfect refusal score isn’t a safer model, Anthropic says
- Anthropic touts 94.2% true positives; curl rejected 19 of 34
- Anthropic’s IPO filing says a key safety threshold is met
- 1,000 Claude agents redid the Fermat proof in 28.7 hours
- Claude Fable 5.2 hits 69% on PCB layout test, up from 25%
Comments
No comments yet. Be the first.
Join the conversation
Sign in to leave a comment. It appears after moderation.

