))
AISN #52: An Expert Virology Benchmark
Manage episode 478465953 series 3647399
Content provided by Center for AI Safety. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Center for AI Safety or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://ppacc.player.fm/legal.
Plus, AI-Enabled Coups.
In this edition: AI now outperforms human experts in specialized virology knowledge in a new benchmark; A new report explores the risk of AI-enabled coups.
Listen to the AI Safety Newsletter for free on Spotify or Apple Podcasts.
An Expert Virology Benchmark
A team of researchers (primarily from SecureBio and CAIS) has developed the Virology Capabilities Test (VCT), a benchmark that measures an AI system's ability to troubleshoot complex virology laboratory protocols. Results on this benchmark suggest that AI has surpassed human experts in practical virology knowledge.
VCT measures practical virology knowledge, which has high dual-use potential. While AI virologists could accelerate beneficial research in virology and infectious disease prevention, bad actors could misuse the same capabilities to develop dangerous pathogens. Like the WMDP benchmark, the VCT is designed to evaluate practical dual-use scientific knowledge—in this case, virology.
The benchmark consists of 322 multimodal questions [...]
---
Outline:
(00:29) An Expert Virology Benchmark
(04:04) AI-Enabled Coups
(07:58) Other news
---
First published:
April 22nd, 2025
Source:
https://newsletter.safe.ai/p/ai-safety-newsletter-52-an-expert
Want more? Check out our ML Safety Newsletter for technical safety research.
Narrated by TYPE III AUDIO.
---
Images from the article:
Apple Podcasts and Spotify do not show images in the episode description. Try Pocket Casts, or another podcast app.
59 episodes
Share
