# AI

Anthropic to Fund Advanced AI Benchmark Development

AI, Anthropic, Claude


  • Anthropic launches a program to fund third-party AI benchmark development.
  • Focus on creating benchmarks for advanced AI capabilities and societal impact.
  • Applications evaluated on a rolling basis; tailored funding options available.


Anthropic Initiates Funding Program to Elevate AI Safety through Comprehensive Benchmarking


In a move aimed at enhancing AI safety and evaluation, Anthropic has unveiled a new program dedicated to funding the development of comprehensive AI benchmarks. Announced on Monday, the initiative seeks to address the growing need for robust AI performance evaluations, particularly for generative models like Anthropic’s own Claude.


Anthropic’s program will provide financial support to third-party organizations capable of creating effective measures for advanced AI capabilities. Applications for funding will be accepted on a rolling basis, ensuring a continuous influx of innovative benchmark proposals.


“Our investment in these evaluations is intended to elevate the entire field of AI safety, providing valuable tools that benefit the whole ecosystem,” Anthropic stated on its official blog. The company acknowledges the challenges in developing high-quality, safety-relevant evaluations and aims to bridge the gap between demand and supply in the AI safety domain.


Existing AI benchmarks often fail to capture real-world usage accurately, raising questions about their relevance, especially those predating modern generative AI. Anthropic’s solution involves creating challenging benchmarks with a focus on AI security and societal implications through new tools, infrastructure, and methods.


Anthropic calls for tests assessing a model’s ability to perform tasks such as cyberattacks, enhancing weapons of mass destruction, and manipulating or deceiving individuals through deepfakes or misinformation. For national security and defense, the company aims to develop an “early warning system” to identify and assess AI risks, though details of this system remain unspecified.


In addition to security-related benchmarks, Anthropic’s program will support research into AI’s potential for scientific study, multilingual communication, bias mitigation, and self-censorship of toxicity. New platforms enabling subject-matter experts to develop evaluations and large-scale model trials involving thousands of users are also envisioned.


The company has hired a full-time coordinator for the program and may purchase or expand projects with scalable potential. While specific funding options were not disclosed, Anthropic promises tailored support to meet the needs of each project.


Despite Anthropic’s commercial interests in the AI race, its effort to support new benchmarks is commendable. The company acknowledges that certain evaluations will align with its AI safety classifications, developed with input from third parties like the nonprofit AI research organization METR.


However, some in the AI community may question Anthropic’s focus on catastrophic and deceptive AI risks, such as nuclear weapons, given the lack of evidence for imminent superintelligent AI threats. Critics argue that such claims divert attention from pressing regulatory issues like AI’s hallucinations.


Anthropic hopes its program will catalyze progress towards making comprehensive AI evaluation an industry standard, aligning with the mission of various open, corporate-unaffiliated efforts to improve AI benchmarks. It remains to be seen whether these efforts will collaborate with an AI vendor driven by shareholder interests.


Buy Bitcoin at Binance

Enjoy up to 20% off on trading fees! Sign up Now!




CoinRank is not a certified investment, legal, or tax advisor, nor is it a broker or dealer. All content, including opinions and analyses, is based on independent research and experiences of our team, intended for educational purposes only. It should not be considered as solicitation or recommendation for any investment decisions. We encourage you to conduct your own research prior to investing.


We strive for accuracy in our content, but occasional errors may occur. Importantly, our information should not be seen as licensed financial advice or a substitute for consultation with certified professionals. CoinRank does not endorse specific financial products or strategies.


CoinRank Exclusive brings together primary sources from various fields to provide readers with the most timely and in-depth analysis and coverage. Whether it’s blockchain, cryptocurrency, finance, or technology industries, readers can access the most exclusive and comprehensive knowledge.