California Weakens AI Safety Bill Before Final Vote, Taking Advice from Anthropic

California’s proposed AI safety legislation, SB 1047, has undergone significant changes following pushback from Silicon Valley. Lawmakers have incorporated several amendments suggested by AI company Anthropic and other critics, softening some of the bill’s toughest provisions.

Key Changes to SB 1047

The revised bill passed through California’s Appropriations Committee on Thursday, marking a major step toward becoming law. Here are the most important amendments:

• The California attorney general can no longer sue AI companies for unsafe practices before a catastrophic event occurs. They can only seek injunctive relief or sue after an incident.
• The proposed Frontier Model Division has been scrapped. Instead, a 9-person Board of Frontier Models will be created within an existing agency.
• AI labs no longer need to submit safety test certifications “under penalty of perjury.” They’ll provide public statements on safety practices without criminal liability.
• Language around safety requirements has been relaxed from “reasonable assurance” to “reasonable care.”
• A $10 million spending threshold exempts smaller developers fine-tuning open source models.

The Rationale Behind the Changes

These amendments aim to address concerns raised by SB 1047 opponents and present a less controversial bill to Governor Newsom. The governor has previously signaled his commitment to California’s AI innovation.

Senator Wiener, the bill’s sponsor, stated: “We accepted a number of very reasonable amendments proposed, and I believe we’ve addressed the core concerns expressed by Anthropic and many others in the industry.”

Mixed Reactions from Industry and Critics

Anthropic is still reviewing the changes before taking a position. The Center for AI Safety Action Fund’s Nathan Calvin said the amendments “will support [the] goal” of advancing AI safety while allowing innovation.

However, some critics remain unsatisfied. Andreessen Horowitz general partner Martin Casado tweeted that the edits are “window dressing” and don’t address fundamental issues with the bill.

Eight U.S. Congress members from California have written a letter urging Governor Newsom to veto SB 1047, arguing it would harm the state’s startup community and scientific development.

Next Steps for SB 1047

The bill now heads to the California Assembly floor for a final vote. If passed, it will return to the Senate for approval of the new amendments. From there, it moves to Governor Newsom’s desk, where he can sign it into law or veto it.

The Bigger Picture: AI Regulation in California

At Social Gains, we’re watching this legislation closely. The outcome of SB 1047 could have far-reaching implications for AI development and safety practices, not just in California but potentially setting a precedent for other states and countries.

While the bill has been softened, it still maintains core provisions holding AI developers accountable for potential catastrophic events. This balancing act between innovation and safety will likely remain a central theme in AI policy discussions moving forward.

What This Means for Businesses

Companies working on large AI models or considering entering the field should pay close attention to SB 1047’s progress. Even with the recent amendments, the bill could introduce new compliance requirements and potential liabilities.

For smaller developers and startups, the $10 million threshold for fine-tuning open source models provides some breathing room. However, it’s crucial to stay informed about evolving regulations in this rapidly changing landscape.

The Road Ahead

As SB 1047 moves through its final legislative hurdles, the AI community and policymakers will be watching closely. The bill’s fate could signal the direction of AI regulation in the U.S. and potentially influence global approaches to managing AI risks and innovation.

At Social Gains, we’ll continue to monitor these developments and provide insights on how they might impact businesses leveraging AI technologies. Stay tuned for updates as this important legislation progresses.