Why Search in China is so Bad
Is it all Baidu's fault? What does it mean for the future of Chinese AI and the broader tech ecosystem?
Welcome to the ChinaEconTalk Newsletter!
In this newsletter I translate articles from Chinese media about tech, business, and political economy. Do subscribe here if you were forwarded this.
I'll be in DC mid-September and NY in early October. Do reply to this email if you live in those cities and would like to meet up!
Search sucks in China. Most blame Baidu—which once was good enough to beat Google in a battle for market share—for search’s demise. Baidu, so the argument goes, prioritized ad revenue and traffic within its ecosystem of sites (Baidu has China’s Wikipedia its own video streaming service) over the best results. However, in the following viral WeChat article, Cao argues that search’s domestic woes run deeper.
Unlike their American counterparts, Chinese tech giants have suffered from more intense competitive pressures so have not learned to play well with each other and thus keep their content proprietary. As a result, China’s internet is now stuck in a walled garden steady state equilibrium akin to the “Dark Forest” of Liu Cixin’s Three-Body Problem.
This default to closed information in the Chinese technology ecosystem has broader implications. Cao argues that the rise of algorithm-based recommendation systems mean that the youth are losing the ability to ask questions and get at the truth. Further, the lack of reliable search results helps fake news spread and walled gardens make it harder for outsiders to innovate and develop sustainable businesses.
This state of affairs may have implications for China’s AI development. Kai-Fu Lee’s assertion that China is the “Saudi Arabia of Data” misses the fact that market competition has prompted leading Chinese AI players to create incompatible independent fiefdoms that are constantly warring at their borders.
What follows is a full translation of the piece.
Search is already dead. Will anyone burn offerings at its grave?
搜索已死，谁来烧纸。
Caozay (Caoz’s Daydreams), July 26, 2019.
In 2002, the first time I listened to Yu Jun [the senior search product manager at Baidu], I thought that the search engine was a revolution that would change human language acquisition. At that time, I thought this point of view was very novel and refreshing. A year or two later, it seemed even clearer to me how right he was. It was no exaggeration to say that search was a revolution.
Recently, some have commented that the decline in Baidu’s market position stems from the fact that the Chinese internet is balkanizing (孤岛化的特征). Taobao and WeChat are increasingly not publishing data to search engines. More and more valuable information cannot be surfaced by search engines. And it’s not just Baidu. Google’s Chinese language search is also suffering.
On the other side, some say that Baidu actually doesn’t care about surfacing links to other companies’ platforms. Instead, they want to create their own ecosystem and ignore the contributions of other content companies.
Regardless, the decline of search engines in China is now irreversible—and being replaced by recommendation algorithms. Instead of engaging in search, users need only click the headlines they find interesting, and new content will be endlessly served up to them.
Some say that many users value the quality of the information they get, and as a result they’ll be constantly served high-quality stuff. Isn’t this a good outcome?
Sorry, but nope. In fact, recommendation logic always runs toward the lowest common denominator.
The logic is very simple. From the perspective of platform creators, click-through and reading rates are most important, so the platforms must appeal to the greatest number of users possible. [See this fantastic New Yorker piece on the fake news produced by Chinese ‘new media’ journalists who studied in the US.] Therefore, even if you’re a super high-level consumer of information, I fear you’ll just end up getting the common man’s content.
Take AlphaGo, for example. When a computer program defeated a human in Go, it was a big deal. Millions of articles were written about the meaning and importance of AI—some alarmist, some sensational, some reasonable. Unfortunately, the deep dives into the science of the feat received far fewer clicks less than media essays. That’s because the click-rate of an academic paper is really low. The audience is too narrow, and thus there are no advertising fees to earn.
The Evils of the Recommendation Era
In the search era, professionals could search for the high-quality content they wanted. Nowadays, in the recommendation era, you’re swept along by the interests of the majority—helpless and unable to do anything but sink under the wave of bad content.
The more insidious evil in this recommendation era is that people are growing increasingly lazy and less likely to ask questions. It’s natural to just lie back and assume the system knows what I want. Young people are no longer searching, no longer used to asking the right questions and seeing good answers.
The golden rule of the search era was that the answer you get depends on the question you ask. In the recommendation era, you don’t need to ask questions or think about problems. You just click on what you find interesting.
Is this a global trend? Actually, no. It’s a “Chinese characteristic.”
Overseas, Google is completely dominant in search, but Facebook and Amazon haven’t walled off their information from Google. In the mobile era, on both Android and Apple phones, search still dominates.
So who is to blame?
Well, it’s easy to blame it all on Baidu, but it’s not 100% their fault. Every techgiant is doing something similar. It would be crazy for Alibaba and Tencent not to maximize traffic.
My guess is that every tech giant has a strong sense of insecurity. If you don’t exert control, others will control you. It’s the prisoner’s dilemma, where no one dares to trust anyone else. Only those smaller companies with this belief can survive China’s brutal market competition, praying that bigger platforms don’t suffocate them.
On the one hand, fierce market competition does promote the prosperity of China’s internet. But this “mutual harm” market structure is also limiting China’s truly leading internet companies.
It’s not like this in the West. For instance, Facebook hasn’t really ever done e-commerce, but the advertising budget of e-commerce on Facebook is astronomical. Google’s Android ecosystem has allowed all mobile phone manufacturers except Apple and Microsoft to use their operating system.
But in China you don’t see this pattern.
Over these past 20 years of China’s internet boom, we’ve done very well and exceeded everyone’s expectations. However, if we can’t face up to the problem and clarify the boundaries of market competition, our hard work may be in vain.
The world shouldn’t have only Dark Forests.
Aside from this newsletter I also host the ChinaEconTalk podcast. This week’s episode with Chris Miller on China’s influence on late Soviet Union policymaking was particularly strong. We discuss how Gorbachev sought to emulate Reform and Opening but was stuck in a sort of middle income trap where too much of the system stood to lose by liberalization. We also look how collapse of the USSR looked to Chinese officials. As Xi himself is reported to have said during a closed-door meeting in 2012:
“Why did the Soviet Union disintegrate? Why did the Soviet Communist Party collapse? … Finally, all it took was one quiet word from Gorbachev to declare the dissolution of the Soviet Communist Party, and a great party was gone. In the end nobody was a real man, nobody came out to resist.”
Please subscribe if you were forwarded this and feel free to reach out directly to me by responding to this email or here on twitter.