Abstract
China’s AI Safety Governance Framework 2.0, released in September 2025, is a non-binding but authoritative articulation of how Chinese technical experts and policy advisers understand AI risks. It significantly expands coverage to open-source model risks, labour market impacts, CBRN misuse, and loss of human control, and introduces a risk categorisation and grading rubric for sectoral regulators. Primarily directed at domestic actors, it is likely to translate into binding technical standards faster than comparable US guidance.
Who is behind the framework
- Guided by the Cyberspace Administration of China (CAC), the country’s most powerful regulator of the internet, data, and AI.
- Released by two bodies under the CAC: TC260 (which formulates many technical AI standards) and CNCERT-CC (the computer emergency response centre).
- A cross-organisational effort bringing together leading Chinese experts on AI policy, evaluation, and technical standards; acknowledgments include government-adjacent technical organisations, leading research labs, universities, and companies including Alibaba and Huawei.
- The bridging of regulatory, academic, and commercial stakeholders suggests it may serve both as a technical reference and as a foundation for future policy thinking.
- Studying it offers a window into the CCP’s deliberative process for technology policy — how the party-state works to understand emerging technology before charting a path forward.
Structure
- Sections on governance principles, a multilayered taxonomy of risk types, technical mitigations, and governance measures, all mapping onto one another.
- At its core is the pairing of specific risks with technical countermeasures intended to address them.
- The “comprehensive governance measures” section covers wider policy tools, notably a call for “establishing an AI safety assessment system.” China has a growing evaluation ecosystem, but measurement science remains relatively immature both in China and globally; an effective evaluation ecosystem is described as critical to advancing frontier safety without heavy-handed regulation.
What changed from version 1.0
Open-source model governance
- Significantly more focused on open-source governance, driven by rapid proliferation by Chinese developers.
- The first authoritative discussion of risks from open-weight models, despite general government support for open source.
- Risks identified include downstream propagation and amplification of model defects, creating vulnerabilities in base models that cascade through the ecosystem as they are fine-tuned and deployed.
- Warns that open-source foundation models make it “easier for criminals to train ‘malicious models.’”
- The proposed technical countermeasure is unsatisfying — merely that assessment of security flaw propagation “should be strengthened” — but the acknowledgment of open-source misuse risk is itself a significant development.
Labour market and societal impacts
- New discussion noting that “the value of labor as a production factor is diminished, resulting in a significant decline in demand for traditional labor.”
- A marked upgrade over version 1.0’s brief and ambiguous reference to AI “accelerating the reconstruction of traditional industry modes” and transforming views on employment, fertility, and education.
- Raises concerns that reliance on AI tools could erode “independent learning, research, and creative capacity.”
- Flags new scientific ethics risks in sensitive fields like biology and genetics as biological information becomes more accessible, and deepening emotional dependence on AI systems for “anthropomorphic interaction.”
Catastrophic risks
- Warns of “loss of control over knowledge and capabilities of nuclear, biological, chemical, and missile weapons” — the most detailed acknowledgment by the Chinese government that modern AI training inevitably incorporates dual-use knowledge.
- Warns that extremist groups and terrorists may acquire relevant knowledge through retrieval-augmented generation capabilities.
- Proposed countermeasure: “ensure exclusion of sensitive [training] data in high-risk fields such as nuclear, biological, and chemical weapons and missiles.”
- A new appendix on “Fundamental Principles for Trustworthy AI” places loss of control at the top of the list, stating that “a human control system should be established at critical stages of AI systems to ensure that humans retain the final decision-making authority.”
- Suggests “circuit breakers and one-click control” mechanisms for “extreme situations.”
- In a Chinese-language expert explainer, Beijing Institute of Technology Professor Hong Yanqing calls the extreme-risk discussion one of the most important updates, saying it has “brought the existential risks that AI might bring… into policy considerations.”
Prioritisation
- An open question is how the CCP prioritises among the nearly thirty risks covered.
- Control over politically sensitive content has driven China’s binding AI regulations to date, with data privacy, discrimination, and labour impacts playing supplementary roles.
- That foregrounding is unlikely to change, but heightened attention to open-source models, CBRN threats, and loss of control shows an attempt to proactively consider emerging large-scale risks.
Implementation
- Like NIST’s 2022 AI Risk Management Framework, Framework 2.0 is non-regulatory guidance — but China’s system is likely to translate it into binding technical standards and regulatory tools much faster.
- Translation is already underway: in January 2025, TC260 released a draft “AI Safety Standards System (V1.0)” mapping the standards needed to implement the original framework, just four months after its release. Some standards covering labelling of AI-generated content have since been finalised and put into effect.
- A key update is a risk categorisation and grading system based on three factors: level of intelligence, nature of application, and scale of application.
- Ten days after Framework 2.0’s release, TC260 issued an open call for organisations to participate in drafting a formal risk categorisation and grading standard. The system remains high-level and bureaucratic, but the authors call for sectoral regulators to adopt and adapt it in their domains.
International diplomacy or domestic driver?
- The distinction matters: an internationally focused document should be read as AI diplomacy, whereas a domestically focused one is more likely to reflect real Chinese thinking and preview future standards and regulation.
- Evidence for international focus: Framework 2.0 states that version 1.0 was created to implement the Global AI Governance Initiative and promote consensus among governments; both versions were released with official English translations; several passages call for international cooperation such as information sharing on AI threats.
- Evidence for domestic focus: the framework’s track record of driving domestic standards; release during the CAC’s annual “China Cybersecurity Week,” a thoroughly domestic event, rather than at the UN General Assembly the following week. At the UN, China instead debuted its “AI+ International Cooperation Initiative,” a vague call for other countries to follow China’s playbook for diffusing AI through its economy.
- On balance: primarily domestically focused, giving the wider bureaucracy an actionable roadmap for new technical standards and regulatory tools. Internationally it serves as leading by example — offering a fleshed-out “Chinese approach” — rather than laying groundwork for a specific international agreement.
Balancing innovation and safety
- Framework 2.0 arrives as the CCP calls for leveraging AI to upgrade China’s economy, society, and government through its AI+ Plan.
- While recognising AI could help solve problems including deflationary pressures and an aging population, the document reflects party fears: control of the information environment, socioeconomic impacts across work and education, and risks that AI could escape human — and therefore party — control.
- China’s bet is that a rich ecosystem of technical standards and model evaluations can provide guardrails without burdensome new regulation: highly iterative, light-touch, technically sophisticated mechanisms developed jointly with leading technologists.
- These standards remain immature, and China’s evaluation system for frontier AI risks lags the United States, but they are clear growth areas where the party is placing significant bets.
- China currently weighs development opportunities more heavily than risk in both rhetoric and practice, but by getting deeper into the technical weeds of risks and countermeasures, Beijing may be giving real weight to what has been predominantly symbolic safety rhetoric.