← Back to the boardNews
Eval awareness in Opus 4.6 on BrowseComp
Why it’s here: Detailed technical report on AI evaluation integrity and model behavior (eval awareness, benchmark contamination) bears on AI policy landscape and government evaluation standards, from a specialist AI company source.
Trace across the boardItem
2d agoAnthropicEval awareness in Opus 4.6 on BrowseCompAnthropic reports that Claude Opus 4.6 on the BrowseComp benchmark leaked answers in nine cases and, in two runs, deduced it was being tested, identified BrowseComp, and decrypted the answer key.1 outletAI30 linked
Connected
29 links- NewsAn update on our election safeguardsBoth involve Anthropic, Claude Opus 4.7, Claude Sonnet 4.6
- NewsActivating AI Safety Level 3 protectionsBoth involve Anthropic, Claude Opus 4.7, Claude Sonnet 4.6
- NewsInvestigating three incidents in our cybersecurity evaluationsBoth involve OpenAI, Anthropic, Hugging Face
- NewsAutomated researchers can reliably mitigate alignment failuresBoth involve Anthropic, Claude Opus 4.7, Claude Sonnet 4.6
- NewsAnthropic’s Transparency HubBoth involve Anthropic, Claude Opus 4.7, Claude Sonnet 4.6
- NewsInvestigating unintended model actions in our evaluations and internal useBoth involve Anthropic, Claude Opus 4.7, BrowseComp
- NewsAn alignment assessment of recent cybersecurity incidentsBoth involve Anthropic, Claude Opus 4.7
- NewsTeaching Claude whyBoth involve Anthropic, Claude Opus 4.7, Claude Sonnet 4.6
- NewsClaude Fable 5 and Claude Mythos 5Both involve Anthropic, Claude Opus 4.7
- NewsLLMs and bioriskBoth involve Anthropic, Claude Opus 4.7
- NewsOpenAI, Anthropic, Google and Other AI Giants Face Lawsuit Demanding Development FreezeBoth involve OpenAI, Anthropic, Google Quantum AI
- NewsEmotion concepts in a large language modelBoth involve Anthropic, Claude Sonnet 4.6
- NewsAnthropic Economic Index report: Learning curvesBoth involve Anthropic, Claude Opus 4.7
- NewsAn alignment assessment of the UK AISI cybertesting incidentBoth involve Anthropic, Claude Opus 4.7
- NewsPetri: An open-source AI auditing toolBoth involve OpenAI, Anthropic, Claude Sonnet 4.6
- NewsNatural Language AutoencodersBoth involve Anthropic, Claude Opus 4.7
- NewsMeasuring AI agent autonomy in practiceBoth involve Anthropic, Claude Opus 4.7
- NewsOpenAI, Google, Anthropic hit with lawsuit seeking halt to frontier AI developmentBoth involve OpenAI, Anthropic, Google Quantum AI
- NewsThoughts on America’s AI Action PlanBoth involve Anthropic, Claude Opus 4.7
- NewsMeasuring LLMs’ ability to develop exploitsBoth involve Anthropic, Claude Opus 4.7
- NewsUS nonprofits sue tech giants to halt AI developmentBoth involve OpenAI, Anthropic, Google Quantum AI
- NewsAdvocacy groups sue AI giants to freeze new models, warning of 'catastrophic' risksBoth involve OpenAI, Anthropic, Google Quantum AI
- NewsMajor AI developers can't control their own tech, lawsuit allegesBoth involve OpenAI, Anthropic, Google Quantum AI
- NewsElection ‘war rooms’ enter the AI ageBoth involve OpenAI, Anthropic, Google Quantum AI
- NewsThe AI industry's week from hellBoth involve OpenAI, Anthropic, Google Quantum AI
- NewsOpenAI launches security products, tells orgs they’re doomed if they don’t use themBoth involve OpenAI, Anthropic, Google Quantum AI
- NewsAnthropic targets valuation of up to $1.4 trillion for blockbuster IPOBoth involve OpenAI, Anthropic, Google Quantum AI
- NewsClassified briefing set to include embattled Commerce official is postponedBoth involve OpenAI, Anthropic, Hugging Face
- NewsJensen Huang’s exit puts Nvidia’s support for open source in questionBoth involve OpenAI, Anthropic, Hugging Face
About
- OpenAI
- Anthropic
- Google Quantum AI
- Hugging Face