I think the nature of every release getting hyped to the moon is what’s driving the takes.
r1 has comparable performance as a prior model OpenAI came out with is less of a sexy headline than, “China is winning! The US messed up!” Just as the idea that Google was triumphant and OpenAI was washed with Veo/Sora lasted all of a couple of days before o3.
Perhaps it’ll be a good thing in spurring more competition and innovation, like Sputnik (… not that boiled down this is anywhere near the same level gap). But I think it’s more likely everyone will lose interest with the next release of whatever…
Respectfully, I have a different perspective on expert controls, though. Export controls may work on small countries like Cuba, which don't have the capacity to develop their own. The notion, however, that export control would hold back China—other than in the short term—seems implausible. China is the second-largest economy in the world, and when measured in PPP GDP, it is the largest economy in the world. All we will do is incentivize the Chinese to develop their equivalent of Nvidia, catch up, and exceed us.
I fear that our 'self-confidence' in our exceptionalism is less and less based on reality (merit) and more and more based on ego.
As I want the US, my home country, to do well, that is not a good development.
Doesn't this make exactly the same mistake that the article points out? It may be true that export controls haven't bitten yet, but that doesn't prove that they don't work or that the counterfactual world without is one where China is not as far behind.
As the article correctly notes, if it was good to have fewer chips then all the US AI companies would want fewer chips, but they don't.
It is remarkable that such reasoning models are internally still implemented as a single Transformer model, that still only produces output token by token, just has a lot more tokens, including "reasoning" tokens, that allow it to do all branching and searching for various strategies while still maintaining the same global context.
At some point it could start forgetting what it did much earlier given that the overall context length is limited. It is also interesting it does not lose its chain of thought given the branching.
Anyhow, looks like the Chinese are catching up fast, though it is a movable goal, and they have yet to show they can actually come with novel ideas rather than replicating what is done in US.
I don’t think anyone doubts that export control “hurts” China. But there is a qualitative difference between “ah I want to hurt you with a punch in your gut” type of hurt or “you are crushed back to Stone Age” type of hurt. It’s obvious that the latter type of hurt is not realised, while this interview really just focuses on justifying the first type, which, yeah, it hurts, but it’s really not the most important conversation.
I agree we should be skeptical that open source will last forever. Certainly could be a marketing tactic.
I'm not sure export controls are entirely not counterproductive. This assumes China won't catch up with chips. If they did and the shoe was on the other foot, would they begin blocking the sale of their chips to the US? Chip Wars says this is probably not possible, but given the track record of doubting China, I wouldn't bet on this.. especially given the tight situation Korea and Japan are in now, not merely their economies but now them realizing the investment they made in the US during Biden may be undone during Trump.. this will possibly make them rethink things.
I think the nature of every release getting hyped to the moon is what’s driving the takes.
r1 has comparable performance as a prior model OpenAI came out with is less of a sexy headline than, “China is winning! The US messed up!” Just as the idea that Google was triumphant and OpenAI was washed with Veo/Sora lasted all of a couple of days before o3.
Perhaps it’ll be a good thing in spurring more competition and innovation, like Sputnik (… not that boiled down this is anywhere near the same level gap). But I think it’s more likely everyone will lose interest with the next release of whatever…
Great article.
Respectfully, I have a different perspective on expert controls, though. Export controls may work on small countries like Cuba, which don't have the capacity to develop their own. The notion, however, that export control would hold back China—other than in the short term—seems implausible. China is the second-largest economy in the world, and when measured in PPP GDP, it is the largest economy in the world. All we will do is incentivize the Chinese to develop their equivalent of Nvidia, catch up, and exceed us.
I fear that our 'self-confidence' in our exceptionalism is less and less based on reality (merit) and more and more based on ego.
As I want the US, my home country, to do well, that is not a good development.
Doesn't this make exactly the same mistake that the article points out? It may be true that export controls haven't bitten yet, but that doesn't prove that they don't work or that the counterfactual world without is one where China is not as far behind.
As the article correctly notes, if it was good to have fewer chips then all the US AI companies would want fewer chips, but they don't.
Editing is for the weak.
Unclear what terrible takes you are referring to. Is it about taking away the export controls due to Deepseek success?
The insight that Deepseek had to innovate around their lack of advanced chips seems obvious to me.
It is remarkable that such reasoning models are internally still implemented as a single Transformer model, that still only produces output token by token, just has a lot more tokens, including "reasoning" tokens, that allow it to do all branching and searching for various strategies while still maintaining the same global context.
At some point it could start forgetting what it did much earlier given that the overall context length is limited. It is also interesting it does not lose its chain of thought given the branching.
Anyhow, looks like the Chinese are catching up fast, though it is a movable goal, and they have yet to show they can actually come with novel ideas rather than replicating what is done in US.
I don’t think anyone doubts that export control “hurts” China. But there is a qualitative difference between “ah I want to hurt you with a punch in your gut” type of hurt or “you are crushed back to Stone Age” type of hurt. It’s obvious that the latter type of hurt is not realised, while this interview really just focuses on justifying the first type, which, yeah, it hurts, but it’s really not the most important conversation.
I agree we should be skeptical that open source will last forever. Certainly could be a marketing tactic.
I'm not sure export controls are entirely not counterproductive. This assumes China won't catch up with chips. If they did and the shoe was on the other foot, would they begin blocking the sale of their chips to the US? Chip Wars says this is probably not possible, but given the track record of doubting China, I wouldn't bet on this.. especially given the tight situation Korea and Japan are in now, not merely their economies but now them realizing the investment they made in the US during Biden may be undone during Trump.. this will possibly make them rethink things.