AI Executive Order: Takeways + Deep Dive

“Virtually the entire federal government has been implicated in this.”

It’s AI week in Washington — and everywhere else! 

To help make sense of things, I recently spoke with the CNAS triumvirate to get their thoughts on the Biden administration’s new fifty-page executive order on AI. We have Vivek Chilukuri, CNAS’s newly minted Director of Technology and National Security, as well as CNAS fellows Bill Drexel and Tim Fist.

Also, LA meetup next week!

Key Takeaways:

Jordan Schneider: This is a long executive order. Tell me one thing about it that excites you.

Vivek Chilukuri: They got eight principles that frame the document, and the majority of them have to do with rights, privacy, civil liberties, and safety. We’re leading by example. That’s important.

Tim Fist: Go government. Go rule of law. They did a thing — some of it seems pretty good. Most of the meat here is in the safety stuff.

Bill Drexel: The talent dimension is most exciting to me — if it works — both in government and in immigration, with the latter being the bigger question.

Jordan Schneider: They just gave themselves so much work. ... Much of this Executive Order will play out over the next year or two, with tons of reports and follow-ups to do in that time frame.

America’s Got Talent?

Jordan Schneider: Let’s look at hiring and immigration. Bill, what do you see?

Bill Drexel: Direct hire for federal government officials with AI expertise is potentially a big win.

This allows the government or various arms of the government to bypass a lot of the red tape that would usually slow down or otherwise make hiring difficult in this particular area.

For such a fast-moving area like AI, it’s important to get people who are well-acquainted with the subject.

Hiring is always a big challenge for the government. Anything that speeds that along is great. AI talent in general is in hot demand in the workforce as well. They have some strong competition, but at least they’re trying to lower the barriers to entry into government in this area.

Vivek Chilukuri: I don’t envision a lot of Hill pushback on this. I don’t think there’s a lot of disagreement on the Hill about the need to bring in AI talent. Congress knows that it needs to do the real work, though, whether that’s H-1B visas or Green Cards and the like.

Jordan Schneider: Direct hire authority is also something the CHIPS Act got to do. It’s one thing for potential government recruits to talk someone into a massive pay cut — but it’s another thing to try to talk someone into a massive pay cut without being able to hire them for seven months or maybe, eventually, not at all.

Bill Drexel: It’s a lot of “consider doing this,” “review this,” and “maybe do that.” But it’s all geared around trying to fix one of the biggest challenges in our AI ecosystem: we get a lot of top talent to come study here, and then we struggle to retain them largely for visa challenges.

This EO is trying to come up with different ways to streamline the process to attract foreign talent and retain foreign talent once they’re already here. It’s just a question of whether — after considering or reviewing these measures — they’ll actually be able to implement changes that stick.

Jordan Schneider: Why can’t the White House just tell departments what to do?

Vivek Chilukuri: These things are always the product of negotiation between the administration and their constituent pieces. I’m actually not sure why they didn’t just direct it outright. They’re providing a menu of options for Congress to direct DHS to do these things outright.

But having not worked directly on executive orders, but a lot of legislative language in the past, you often have agencies that are pleading for flexibility and prefer to be asked to consider something rather than directed to do it. I suspect the admin might have lost a few battles with DHS on this one. DHS has a lot on its hands right now.

Jordan Schneider: Getting more STEM talent by rejiggering administrative regulation seems to be the most straightforward thing. I was a bit disappointed there wasn’t more in there. The most obviously stupid thing is the US’s lack of domestic visa renewal. Having to leave the country every time you want to get your visa is hugely annoying and disruptive.

Vivek Chilukuri: With all the volatility in the tech sector, at least in the last eighteen months, you’ve had a lot of people with interruptions in their employment, even though they’re highly employable in their sectors.

I’m pretty sure Canada earlier this year introduced a visa program directly targeting US H-1B holders.

We’re just sort of hemorrhaging talent through the dysfunction of our immigration system. It’s against our own interests.

AI Safety: We Can Act If We Want To

Tim Fist: This EO is heavy on AI safety and security.

But it’s the Commerce Department that’s being asked to do a lot of the stuff in this area. Commerce is increasingly emerging as the default place where AI safety regulation is happening.

The EO is also pretty focused on what they term “dual-use foundation models.” The report defines this kind of model:

The term “dual-use foundation model” means an AI model that is trained on broad data; generally uses self-supervision; contains at least tens of billions of parameters; is applicable across a wide range of contexts; and that exhibits, or could be easily modified to exhibit, high levels of performance at tasks that pose a serious risk to security, national economic security, national public health or safety, or any combination of those matters.

The EO offers a bunch of technical definitions for how they might define these models in terms of the computation required for training. But then they also ask Commerce to come up with a better definition.

Their initial definition is any model that uses more than 10^26 operations for training. That’s a whole lot of training compute. OpenAI’s GPT-4 — the most powerful model on the market today — probably used only about a fifth of that level of compute.

There’s also a distinct, separate threshold specifically for models that primarily trained on biological sequence data — models that could be used to develop new synthetic viruses, for example. There’s a much lower compute threshold, for that; it’s about 1,000 times lower.

There are also corresponding limits around the computing infrastructure linked to those. For that, it’s basically 10^20 operations per second, which you can think of as being around 20,000 of Nvidia’s latest H100 GPUs in a big cluster.

Bill Drexel: 

Biosecurity is the preeminent AI safety challenge. The EO has a fair bit on cyber as well, but biosecurity seems to be what’s on a lot of people’s minds. These things could pose a threat to public safety or even enable bioterrorism.

I give them some kudos for being proactive and mentioning that we can use AI to boost our biosecurity defense proactively and not just look for ways to shut it down.

A recent paper by an MIT professor recently looked at how Meta’s open-source Llama 2 model could potentially be used to help people create dangerous pathogens, including the 1918 Spanish flu.

Biology is a lot more complicated than chemical design tools, so we would need bigger models. But in theory you could get some biological tools that are not that large (in terms of parameters) that could still do some nasty stuff.

Jordan Schneider: I am personally banking only on the good AI outrunning the bad when it comes to biosecurity. But that framework might make a little more sense in the cyber context.

Tim Fist: There are a few different measures addressing biosecurity.

1. One is a compute threshold for reporting.

2. The second one is they’re commissioning a report on bio risks.

3. The third one is the most interesting here: it introduces a screening process for providers who are actually producing synthetic nucleic acids. This is like synthetic DNA that you can use to create viruses.

This EO creates measures for companies that are actually providing the inputs needed to carry out a bio attack. These companies will actually have some sort of screening and regulation on what they’re providing customers and what they can do with it.

Bill Drexel: Bad actors can possibly create conventional bioweapons more easily, but that usually requires foundational models that need a whole lot of compute. At the moment, there’s nothing these models can do that you couldn’t already do with the Internet. They just make it faster. Even then, they introduce mistakes.

The other issue is biological design tools that will eventually be able to design pathogens with greater precision and potentially with the ability to circumvent a lot of the screening tools.

Jordan Schneider: How does this EO approach concerns about foreign access to cloud computing when it comes to training models?

Vivek Chilukuri: The federal government, at a minimum, is doing what it usually does:

The US government is collecting data, getting a sense of the landscape, and building muscle memory within the public and private sectors to potentially lay the groundwork for more explicit restrictions on cloud computing down the road. This is something that was withheld from the last export controls update.

The top one is GPT-4.

Après Moi, le Déluge

Jordan Schneider: How does the fact that there will be lots of reports coming down the pipeline change the sort of impetus for action on the Hill on these topics?

Vivek Chilukuri: Over the next 540 days, everybody’s just going to be writing reports and updating standards and considering whether to implement new programs. 

But look … Hill staffers often read these reports. They’re often briefed. This is an opportunity for organizations like CNAS and others to bring the government and Hill staff together as they’re developing these reports. It’s an opportunity to help them think through their contents.

Still, it’s a massive amount to track and it’s going to be a lot for the Hill to swallow in terms of keeping all of these pieces in their heads.

But the breadth of coming work means that virtually every committee will have an interest in this. Virtually the entire federal government has been implicated in this.

There are probably some agencies whose reports have more weight and credibility, and the Hill may be less interested in reviewing reports from other agencies.

The separation of powers also means the Hill will do its own homework. They want to have an independent perspective on these issues. They’re not necessarily going to defer to the Department of Defense or the Department of Commerce.

And sometimes agency reports are just written by a committee, and don’t actually push the frontier of thinking or inform people beyond more than they know.

People respect the reports the US Intelligence Community publishes. I don’t want to pick on anybody — but I can’t imagine people are going to have high expectations for agencies like the Department of Labor or Health and Human Services, which don’t have much history or personnel in this area.

Jordan Schneider: There’s all this stuff in here which is impossible and unknowable. And I’m not sure that, in 270 days, anyone’s going to be able to come up with a great framework for what to do about open source — because the next 270 days are going to be way different from the past 270 days in terms of how capable these things are.

But one thing you can be moderately certain about is that government has the potential to be radically improved in terms of day-to-day productivity. Within sixty days, each agency must have a chief artificial intelligence officer. These officials are going to be stressed out about risks while also evangelizing AI tools and regulatory flexibility in daily work.

That could be a great way to improve how the government works. You may not be saving the world from a global AI pandemic, but someone’s got to process the visas.

Vivek Chilukuri: It’s easy to appoint people like that in agencies, but if they’re not empowered and they’re not reporting to the right people, it can just be a nominal exercise.

Seeing Red (Teams)

Jordan Schneider: If you’re training GPT-5, the US government now politely requests that you share your red-teaming results, what you’re doing with the weights, and how you’re setting up your cybersecurity structure. But does that matter at all?

Tim Fist: What if a red team discovers that a company has developed a model that allows them to carry out large-scale novel bio attacks. What do we want to happen?

As far as I can tell, the stuff in this executive order doesn’t actually give government the power to block a company from releasing that model or even open-sourcing it. It’s just more reporting about how they went about training and risk management.

The Defense Production Act (DPA) “provides the President with an array of authorities to shape national defense preparedness programs and to take appropriate steps to maintain and enhance the domestic industrial base.”

Typically, this is about giving the government the goods and making it a critical buyer for defense items needed by the government; it was used during COVID to get things like respirators and masks to the right place. But it’s interesting that it’s being used here for regulatory purposes here.

It’s unclear to me whether you can use it to block companies from doing specific things.

Vivek Chilukuri: My understanding of the DPA is that the US government can compel companies to produce goods for national security purposes. I’m not aware of any administration using the DPA to block production. That doesn’t mean that they can’t use it in that way, but I was honestly a little surprised to see it here.

Tim Fist: What you can imagine doing is having regulators make reporting requirements so generous that you in fact block the release of a model because of the costs of compliance.

Jordan Schneider: What if my red-team activity is trash? Or what if I don’t have great cybersecurity?

There’s no enforcement mechanism here.

We’re going to get all these AI companies to show up to the White House and tell them to play nice and sign on to all these principles. That’s great — until there’s someone who has enough money and compute and who doesn’t feel like the norms apply to them.

Maybe after 270 days we’ll have a sharper stick. 

Tim Fist: 

These requirements set up monitoring and evaluation that get us the data needed to make additional moves. This is all they’re allowed to do under an executive order — trying to solve that measurement problem is still a fairly great idea.

Vivek Chilukuri: There might be some virtue in taking our time and collecting and getting a sense of the state of play — whose red teams are actually doing the work and whose are just phoning it in.

Roam Around the AI World

Jordan Schneider: This week we have the AI Safety Summit in the UK. What’s going on here and around the world?

Bill Drexel: This is just a couple of weeks after China released its vision for international AI governance.

The end of this executive order also talks about how the EO sets the stage for what America will do internationally.

The AI Safety Summit seems like the beginnings of the flagship attempt by like-minded countries to establish guardrails for AI. The London summit has a much narrower set of concerns, mostly to do with catastrophic risks. The summit also has a major focus on biosecurity.

It’ll be interesting to see the extent to which China is interested in these concerns and controls that the US and the UK are suggesting.

It seems like China is trying to position itself as the leader of the Global South. It has consistently criticized the US groups that want to contain the technology. China contrasts itself as a power that will allow developing countries to seize the moment in AI.

The AI Safety Summit does have a pretty diverse attendee list. It integrates countries that are not exactly technical leaders in AI.

Jordan Schneider: I’m of two minds, because China clearly sees something in this. I’m sort of surprised by it. I’m sort of surprised in general — the idea that the whole world is excited about global AI governance.

Unlike the invasion of Ukraine or other depressing geopolitical problems, China seems open to having discussions about AI governance. But it would be weird if AI were the one thing that broke the mold of China’s past diplomatic efforts.

I do wonder, however, why the White House is boasting that this EO is the most comprehensive action by any government when it comes to AI. What does that say about the Biden administration’s ambitions?

Tim Fist: The EO has some language about wanting to expand engagements with allies, bilaterally and multilaterally, including by attending this AI Safety Summit.

They also talk about wanting to take the voluntary commitments that large US labs have signed and expand those internationally.

The White House sees these existing AI safety and security principles as the vehicle for what they want to push globally. The other thing is this set of technical standards for AI risk management.

Vivek Chilukuri: You need a principles-based regulatory approach. If you have fixed standards and specifications, they’re just going to get outpaced within 270 days.

Tim Fist: Standards by themselves provide a sort of soft law basis for global regulations. The goal may be to start off with something like a National Institute of Standards and Technology risk-management framework for AI, which seems fairly high-level at the moment.

But there’s a version they’re developing specifically for generative AI which can be made global. The International Organization for Standardization can then turn these into technical standards the rest of the world can agree on. Future global regulations could then be based on that. They want to define some of these concepts first.

Bill Drexel: I agree with Tim. It seems as though these voluntary commitments are what they’ve explicitly said they want to internationalize, but at least their rhetoric has been trying to internationalize the whole package.

They’re trying to pioneer a pro-democracy approach to AI — which contrasts with China’s export of techno-authoritarianism and the Xinjiang model being provided. I doubt the area where China and the US’s stated objectives have the most overlap is on these frontier model issues.

To the degree that there is this weird race to being the one to win the regulatory discussion internationally, I suspect it’s going to be to what degree can we get either pro-democracy or pro-autocracy provisions established, if any. I doubt the area where China and the US’s stated objectives have the most overlap is on these frontier model issues.

To the degree that there is this weird race to being the one to win the regulatory discussion internationally, I suspect it’s going to be to what degree can we get either pro-democracy or pro-autocracy provisions established, if any.