AISI is supposed to protect the United States from unsafe AI models by conducting security tests to detect harm before the models are deployed. Testing should “address risks to human rights, civil rights and civil liberties, such as those related to privacy, discrimination and bias, freedom of expression and security of individuals and groups,” President Joe Biden said. said in a national security memo last month, insisting that security testing was essential to supporting unparalleled AI innovation.

“For the United States to benefit most from AI, Americans need to know when they can trust the systems to work safely and reliably,” Biden said. said.

But AISI safety tests are voluntary and, even if companies as OpenAI and Anthropic agreed to voluntary testingthis is not the case for all companies. Hansen is concerned that AISI lacks the resources and budget to achieve its overall goals of protecting America from the untold harms of AI.

“The AI ​​Safety Institute predicted they would need about $50 million in funding, and that was before the national security memo, and it doesn’t look like they’re going to get that amount at all,” Hansen told Ars.

Biden had budgeted $50 million for AISI in 2025, but Donald Trump did so. threatened to dismantle Biden’s AI security plan upon taking office.

AISI will likely never be adequately funded to detect and prevent all the harm caused by AI, but with its future uncertain, even the limited safety testing planned by the United States could be blocked at a time when the industry of AI continues to advance at full speed.

This could largely leave the public at the mercy of AI companies’ internal security testing. While big companies’ pioneering models will likely remain under society’s microscope, OpenAI has promised to increase investment in security testing and help establish industry-leading security standards.

According to OpenAI, part of this effort includes making models more secure over time, less likely to produce harmful output, even with jailbreaks. But OpenAI has a lot of work to do in this area, as Hansen told Ars that it has a “standard jailbreak” for OpenAI’s most popular version, ChatGPT, “that almost always works” to produce harmful outputs.