Here are two claims I hear about China and AI safety:
China is intrinsically less concerned about AI risk. Chinese people are techno-optimists, while the Party sees doom talk as either Western paranoia or a containment strategy designed to hold China back — such as many of the dismissive responses to Dario’s call to pace the frontier.
China is now safety-pilled. This refrain gains traction once or twice a year when a high-up like Xi gives a speech about controlling AI risks (as he did at the World AI Conference), or China puts forth a new international initiative, such as the September UN Global Dialogue on AI Governance.
There’s evidence for each of these takes (see footnote).1 But I think both sides miss the messier reality on the ground.
I met plenty of ambitious and talented people when I lived in Beijing who would have loved to work on AI safety full-time. But a graduate in China who cares about AI risk is still expected to get a “real” job eventually. Outside an underfunded PhD track or a handful of positions in a country of 1.4 billion people, there isn’t a well-trodden path. In San Francisco, DC, or London, worrying about AI professionally can be a very-well-paid career. In China, it’s more of a hobby.
As a result, I think the main bottleneck facing China’s AI safety ecosystem is not ideological but material…
Independent Chinese AI safety has basically no money.2 And the limiting funding structures that do exist determine what kinds of safety work can survive, and where.
Today:
Why China’s philanthropic rules make it difficult to fund independent organizations, even if the money exists.
Why this leaves it up to the Chinese government to lead on safety work
Technical safety research and how it is continuing to grow
China’s “safety-as-a-service” companies which try to make safety profitable
A “third wave of American philanthropy” appears to be on the horizon, alongside initiatives like Coefficient Giving’s Project Tailwind, potentially enabling much more ambitious AI safety efforts in the West. Understanding how this influx of resources could reshape the gap between Western and Chinese AI safety ecosystems (and the relationship between them) therefore seems increasingly urgent.
[Nathan Labenz explores related themes in his dispatch from China, which I’d recommend checking out.]
A Brief Overview of Chinese Philanthropy
China has plenty of billionaires. Forbes puts the number at 539 billionaires in China worth $2.2 trillion. (The US counts 989 billionaires worth $8.4 trillion.)
It would take only one or two of them getting really into AI safety to finance a sizable funding ecosystem. So why has no Chinese equivalent of Coefficient Giving emerged?
The answer is related to the strange structure of Chinese philanthropy.
The modern Chinese charitable sector as it exists today expanded rapidly after the 2008 Sichuan earthquake, but it was plagued from the start by corruption scandals, inconsistent regulation, and government concerns about grassroots organizations operating outside of the state’s purview.
To streamline the system, the 2016 Charity Law 慈善法 clarified the legal status of domestic philanthropy, tax incentives, and fundraising. Charities would have to register, disclose their finances, and accept government supervision.
The goal wasn’t to reduce or get rid of philanthropy. Beijing still wanted private money to support education, health, poverty relief, and other social services. But it wanted civil society to help deliver state-approved public goods, rather than become an independent source of political organization or agenda-setting. Article 4 of the Charity Law made this principle explicit enough by stating that “charity work shall be conducted under the leadership of the Communist Party of China.”3
Scholars Angela Bies and Scott Kennedy write:
“There exists a current paradox of philanthropy and the state in China: a stronger social sector accompanied with growth in philanthropy is desired, but such strength and growth must also contend with the state’s own policy agenda and desire to limit aspects of advocacy and civil society typically associated with the third sector.”
The Charity Law may have legitimized the sector, but it has not erased longstanding distrust. China has long ranked near the bottom of the Charities Aid Foundation’s World Giving Index, which measures behaviors including donating money and volunteering. In off-the-record conversations for this piece, I likewise heard skepticism about donating due to corruption.
^The 2011 Guo Meimei 郭美美 fiasco, one of many scandals that badly damaged public confidence in major charities.
Perhaps some of this lingering distrust also reflects another impetus behind the 2016 reforms: limiting the influence of foreign organizations operating inside China — such as groups focused on human rights. This led to the Overseas NGO Law 境外非政府组织境内活动管理法 (2016), which, for AI safety, may actually be more consequential than the Charity Law.
The Overseas NGO law mandates that foreign nonprofits must register an office or work through an approved Chinese partner, with the system overseen by public-security authorities. Shawn Shieh, a research associate at the British charity ODI, found that many Chinese non-profits went from receiving 80–90% of their funding internationally to 80–90% from domestic Chinese sources in the years following the Overseas NGO Law, effectively flipping the philanthropic funding ecosystem on its head.
This should help clarify why supporting AI safety work in China is unlikely to be as straightforward as simply giving more money to Chinese non-profits to work on safety. In practice, foreign funding has to flow through organizational structures that are acceptable to the government. Even if Beijing fully accepted the case for addressing AI risks, it would still be unlikely to permit foreign funding on a large scale to flow into Chinese organizations.
Chinese charitable organizations received roughly $21 billion in donations in 2023, compared with $557 billion in the US. That’s a 27x difference.
Companies supplied 77% of Chinese giving in 2023, while individuals provided only 22%. In the US, the pattern is reversed. Individuals supplied 67% of charitable giving in 2023, while corporations accounted for just 7%, with foundations providing another 19%.
Specifically for AI, I estimate Western AI safety philanthropy is $250-600 million per year.4 And I predict that number will push well beyond $1 billion by next year, after companies like Anthropic and OpenAI go public and usher in a potential “third wave” of American philanthropy.
What Chinese money does exist flows toward established causes. Health received 37% of donations in 2023, education 25%, and scientific research around 10%. Among the country’s largest philanthropists, 70% donated to education. Harvard’s China Philanthropy Project similarly finds that education dominates elite giving. This fits the logic of the Charity Law, which encourages philanthropy in politically approved areas already recognized as socially useful.
State-Sanctioned Safety
Perhaps in part because of these restrictions on charitable giving, AI safety in China tends to be treated less as a philanthropic domain than as a responsibility of the state. China’s largest AI safety institutions are government or government-backed, like Shanghai AI Lab (SHLAB), the Beijing Academy of Artificial Intelligence (BAAI), the China Academy of Information and Communications Technology (CAICT) and TC260.
None of these institutions primarily focus on safety. CAICT is a massive Ministry of Industry and Information Technology (MIIT) think tank. BAAI trains models for capabilities, such as Emu3.5. Safety is one function inside a much broader mission, competing for resources with everything else that mission entails.
^BAAI’s Emu3.5. Looks interesting but doesn’t seem to have anything to do with safety.
Shanghai AI Lab (SHLAB) comes closest to an exception. SHLAB has a dedicated Safety and Trustworthy AI program, accounts for roughly one in ten papers in Concordia’s Chinese frontier-safety database, and this year released the Shu’an (书安) AI safety platform. But it is also working on capabilities, shipping the InternLM (书生) model family.
Zhou Bowen 周伯文, who runs SHLAB, also chairs TC260’s AI Safety Standards Working Group (WG9), arguably the most consequential AI safety body in China. But the number of positions inside a state-sanctioned body like TC260 is small.5 And the work itself is different in kind from an independent safety ecosystem.
When I first became interested in AI safety in 2019, ideas that are now somewhat mainstream still sounded pretty sci-fi, and certainly not politically tractable. But that distance allowed researchers to chase strange or unpopular questions without first clearing them against political coalitions, geopolitical competition, or bureaucratic consensus.
The independent US/Western ecosystem still preserves some of that freedom. A smart young researcher can leave college, join a small nonprofit, and spend years pursuing an idea that neither Washington nor a frontier lab currently considers important. A state-centered ecosystem is inevitably more credentialed and hierarchical, which makes it harder to build the same kind of open career field.
To be clear, I’m not saying this makes China’s state-backed work unimportant. WG9, SHLAB, and others are doing very impactful work! But government attention cannot fully substitute for independent organizations capable of nurturing ideas before the state decides they matter.
Academic/Technical Work
The independent safety organizations that do exist in China are generally thinly staffed or financially opaque. CnAISDA, China’s self-described counterpart to AI Safety Institutes elsewhere, launched in February 2025 not as a new organization with its own budget and staff but as a network stitched together from people already employed at Tsinghua, BAAI, CAICT, and others. The Beijing Institute for AI Safety and Governance similarly draws heavily on researchers whose main institutional homes are elsewhere. What these orgs have in common is that they sit inside academic institutions, which seem to have carved out a workable space for doing safety research.
Concordia AI published its 2026 State of AI Safety in China report alongside an interactive database of Chinese frontier safety papers, bringing the state of technical research in China into focus. Chinese institutions published around 12 frontier safety papers per month in 2023, roughly 26 by mid 2025, and 57 by April 2026, nearly a fivefold increase in two years.
In certain subsets of safety research — like agents governance — China is pumping out more research than anyone.
To be clear, American authors still dominate the field’s most-cited work, and safety accounts for a relatively small fraction of China’s overall AI publications (see footnote).6 The big results that actually push the field’s agenda forward each year, like alignment faking, circuit tracing, the AI control paradigm, still come almost entirely out of Silicon Valley (with Anthropic being the main contributor).
But this is also influenced by funding structures. Anthropic and OpenAI can afford to run dedicated interpretability and alignment teams because compute is not the binding constraint on their existence, whereas a Chinese lab burning every available H-equivalent hour just to stay competitive on capabilities has less comparable slack.
Most of China’s AI safety research therefore comes out of universities and state-backed labs rather than companies. Of the 28 “Key AI Safety Research Groups” identified by Concordia, 20 are academic. In the West, by contrast, frontier labs play a much larger role in both producing safety research and setting the field’s agenda.
“Safety-as-a-service” Companies
Putting everything above together, China’s funding structures also help explain what I think of as its “safety-as-a-service” companies. These firms do work that could reasonably count as AI safety, but package it alongside cybersecurity, regulatory compliance, and other commercial products and services.
Scholars Wenjun Lai and Anthony Spires use the term “venture philanthropy” to describe how China’s philanthropic constraints often push public-interest work toward measurable outputs and business-like models. This seems similar to what these safety-as-a-service companies are offering.
As I covered previously, China’s generative-AI filing regime created a market for companies that help developers pass government review. Firms such as RealAI, BotSmart, and Qihoo 360 have tried to use that demand as a way to sneak in some more robust safety services like red-teaming, robustness evaluations, cybersecurity testing, and ideological compliance assessments, which are heftier topics than just compliance with the CAC.
Since I wrote that piece, the evidence that this is becoming a real market has gotten stronger. The number of officially filed generative-AI services increased from 439 in June 2025 to 988 in June 2026. The market is also becoming more institutionalized. Beginning in 2025, MPS began certifying products such as large-model security guardrails and evaluation systems.
More importantly, customers are actually buying dedicated safety products.7
But what hasn’t changed is that these companies cannot operate like Western safety nonprofits whose central deliverable is simply “make advanced AI safer.” They will only investigate the safety problems for which a customer is willing to pay. But that means the customer has to be willing to pay. Money is, once again, what it comes down to.
What’s to be done?
Is the conclusion to all this simply that the US-China safety funding gap will just keep getting larger? I’d be lying if I didn’t say maybe. The Overseas NGO Law makes direct Western grantmaking into China tough, while Western funders sending money to state-linked institutions raises obvious political and security concerns.
Still, here are some possibilities.
Funders could establish things outside of China that are valuable to Chinese researchers. Pacific Compute is already experimenting with this model by putting safety-dedicated GPUs in Singapore that AI researchers can access for evaluations and research — emphasizing Chinese safety researchers in particular. Something similar could work in other third countries, giving Chinese researchers access to safety resources without transferring the resources into China themselves.
Exchange programs. Funders could help researchers spend time working together elsewhere. Or they could finance international researchers going to China, such as the Singapore AI Safety Fellowship.
Invest instead of doing philanthropy. Foreign investment in Chinese companies operates under a different legal regime from foreign NGO funding, although both Chinese restrictions and US outbound-investment rules can both still bite.8 Finding ways to support commercially viable safety companies (perhaps those doing “safety-as-a-service”) may be more realistic than independent nonprofits, though there is the risk that these companies will care more about profit and their shareholders than safety as an inherent goal.
The longer-term solution may be to fill the gap inside Chinese philanthropy itself. China has plenty of wealthy donors, but relatively few grantmakers whose job is to hunt for problems and then give researchers discretion to pursue them. A domestic AI-safety fund or regranting institution could pool Chinese money and distribute it to researchers and small organizations. It could also develop a direct relationship with the Chinese government and Western donors, turning this into a streamlined procedure rather than a bespoke process done by a bunch of smaller nonprofits. For instance, a Western funder could, in theory, register a permanent representative office, which requires a Chinese government sponsor and approval from provincial public security authorities. The Gates Foundation and other big non-profits have taken this route, though it seems arduous.
Most of these proposals are far-fetched, but so are essentially all of the proposals for global AI cooperation on AI (like AI 2040’s Plan A). And I doubt good cooperation can be conjured into existence at the moment of crisis. It will need a Chinese safety ecosystem with enough independence for new ideas to grow and enough resources for talented people to spend their careers pursuing them.
And that requires some cash.
*Nick is a visiting fellow at the Carnegie Endowment for International Peace.*
Survey data consistently finds Chinese respondents much more optimistic about AI than Americans, and that gap may actually be widening as US attitudes continue to nosedive. The 2026 Ipsos AI Monitor, for instance, found 83% of Chinese respondents saying AI’s benefits outweigh its drawbacks, compared with 39% of Americans (the biggest gap I’ve ever seen). But survey data only tells us so much, and we have written previously about how AI anxieties manifest differently in China.
Rhetoric also matters. When Xi talks about controlling AI risks, it changes what officials and researchers feel permitted to prioritize. Even if some of it is hot air, I still wouldn’t mind if Trump at least acknowledged these risks aloud. There have also been signs that Chinese officials may be taking certain risks more seriously, including discussions of cybersecurity risks, restricting overseas access to advanced models and various regulatory moves, such as TC260’s Safety Governance Framework 3.0.
Jasmine Li estimates in this post that annual philanthropic funding for frontier AI safety is around $20 million in China, compared to $1 billion+ in the rest of the world. A funder with expertise in the space gave me the same estimate.
What this means in practice is vague, but I think it’s unlikely a Party committee signs off on every donation. Organizations register with civil-affairs authorities, submit annual work and financial reports, and may operate under a supervisory agency. Separate Party rules for social organizations call for Party organizations to be established inside eligible nonprofits, allow Party bodies to vet organizational leaders, and encourage Party secretaries to participate in management meetings.
Coefficient Giving alone said its technical AI safety team expected to make more than $130 million in grants in 2025, while its AI Governance and Policy team moved more than $140 million.
According to Claude: TC260's member directory currently lists 311 member units under WG9, out of 704 total TC260 member units. For comparison, the data security working group has 328 and SWG-ETS (the emerging technology security special working group) has 202. Source.
For instance, the 2025 Global Index for AI Safety, using DBLP data through February 2025, counted 9,521 Chinese AI-safety publications and 9,319 American ones — essentially a tie. But China’s overall AI research output is much larger. The 2026 Stanford AI Index estimates that China produced 17.8% of global AI publications in 2024, compared with 7.3% for the US. On these measures, China produces roughly the same volume of safety research from an AI research base about 2.4 times larger, suggesting that safety constitutes a smaller share of Chinese AI research overall. Exact figures vary across datasets and definitions of “AI safety,” but this is the general pattern I have observed after looking at multiple sources.
Postal Savings Bank, UnionPay International, and Guangfa Bank have all procured large-model safety evaluation or guardrail systems. Z.AI has also publicly confirmed using NetEase for dangerous-capability evaluations and training-data audits, suggesting frontier labs themselves may outsource some safety work. Meanwhile, IDC now treats agent-threat detection as a distinct Chinese market, and RealAI raised several hundred million RMB this July to expand its safe-and-trustworthy AI business; Xi visited them earlier this year also and they increasingly discuss frontier risks such as deception, self-replication, and uncontrolled AI R&D.
For instance, US rules that prohibit outbound investment to “countries of concern” would likely apply to investments to Chinese AI companies, while China rules subject foreign investment of any kind to its own access rules and national-security review.
The safety-as-a-service route raises a procurement question: who pays for findings that delay a launch? If the buyer can suppress an unfavourable evaluation, more safety spending could still produce little public knowledge. Could funding include a protected right to publish negative results, even when the customer walks away?
What I find most interesting here is that AI safety isn't just a technical problem — it's also an institutional one. The question of who funds safety research, and what those funders actually want from it, can shape which safety questions get studied in the first place.