Implications and Solutions for AI Bot Traffic on Library Websites with Rosalyn Metz

The Authority File

This month’s episode features Rosalyn Metz, Chief Technology Officer for Libraries and Museum at Emory University. At a 2024 Charleston Conference panel with TAF host Bill Mickey, Rosalyn shared a quick anecdote on how spikes in AI bot traffic on Emory library’s website have lead to service shutdowns. Now, Rosalyn joins the podcast to dig into this frustrating issue, exploring how her library has dealt with this traffic and the ethical implications of making a library’s catalog and metadata openly available on the Internet in the age of AI.

To start, Rosalyn explains the level of traffic the library has experienced in the past year and how staff have worked to distinguish between regular bot traffic and large language models crawling for content. Further, she chats about the protections her library and other institutions have instigated, whether that be turning to campus IT, seeking out vendor partnerships, or locking down library webpages completely. Sharing her concerns beyond interrupting site traffic and the user experience, Rosalyn discusses the implications of AI bots scraping unvetted institutional files and sensitive repository materials.

Missed an episode? Subscribe to our monthly newsletter, Choice Podcast Updates, and check out the Authority File Round-Up on our blog, Open Stacks!