When RAND Made Magic + Jason Matheny Response
Did the magic stop, or just get classified?
Our article was originally published in Asterisk Magazine. Today, ChinaTalk is rereleasing it alongside exclusive commentary from Jason Matheny, CEO of RAND at the end of the post.
RAND’s halcyon days lasted two decades, during which the corporation produced some of the most influential developments in science and American foreign policy.
So how did it become just another think tank?
Between 1945 and 1960, RAND operated as the world’s most productive research organization. Initially envisioned as a research arm of the Air Force, RAND made century-defining breakthroughs both in basic science and applied strategic analysis. Its members helped define U.S. nuclear strategy, conceptualized satellites, pioneered systems analysis, and developed the earliest reports on defense economics. They also revolutionized much of STEM: RAND scholars developed the basics of game theory, linear programming, and Monte Carlo methods. They helped conceptualize generalized artificial intelligence, developed the basics for packet switching (which enables data transmission across networks), and built one of the world’s first computers.
Today, RAND remains a successful think tank — by some metrics, among the world’s best.1 In 2022, it brought in over $350 million in revenue, and large proportions still come from contracts with the US military. Its graduate school is among the largest for public policy in America.
But RAND’s modern achievements don’t capture the same fundamental policy mindshare as they once did. Its military reports may remain influential, but they hold much less of their early sway, as when they forced the U.S. Air Force to rethink several crucial assumptions in defense policy. And RAND’s fundamental research programs in science and technology have mostly stopped. Gone are the days when one could look to U.S. foreign policy or fundamental scientific breakthroughs and trace their development directly back to RAND.
How was magic made in Santa Monica? And why did it stop?
The Roots of RAND
Economists, physicists, and statisticians — civilian scientists to that point not traditionally valued by the military — first proved their utility in the late stages of World War II operational planning. American bomber units needed to improve their efficiency over long distances in the Pacific theater. The scientists hired by the Army Air Force proposed what at the time seemed a radical solution: removing the B-29 bomber’s armor to reduce weight and increase speed. This ran counter to USAAF doctrine, which assumed that an unprotected plane would be vulnerable to Japanese air attacks. The doctrine proved incorrect. The increased speed not only led to greater efficiency, it also led to more U.S. planes returning safely from missions, as Japanese planes and air defense systems were unable to keep up.2 Civilian scientists were suddenly in demand. By the end of the war, all USAAF units had built out their own operations research departments to optimize battle strategy. When the war ended, the question turned to how to retain the scientific brain trust it had helped to assemble.
General Henry “Hap” Arnold, who had led the Army Air Force’s expansion into the most formidable air force in the world, had started to consider this question long before the war had ended. He found an answer in September 1945, when Franklin Collbohm, a former test pilot and executive at Douglas Aircraft, walked into Arnold’s office with a plan: a military-focused think tank staffed by the sharpest civilian scientists. Collbohm did not have to finish describing his idea before Arnold jumped and agreed. Project RAND was born.
Arnold, along with General Curtis LeMay — famous for his “strategic bombing” of Japan, which killed hundreds of thousands of civilians — scrounged up $10 million from unspent war funds to provide the project’s seed money, which was soon supplemented with a grant from the Ford Foundation. This put RAND into a privileged position for a research organization: stably funded.
On top of that financial stability, RAND built what would become one of its greatest organizational strengths: a legendarily effective culture, and a workforce to match it.
Internal Culture and Talent
In an internal memo, Bruno Augestein, a mathematician and physicist whose research on ballistic missiles helped usher in the missile age, highlighted a set of factors that catalyzed RAND’s early success. In short: RAND had the best and brightest people working with the best computing resources in an environment that celebrated excellence, welcomed individual quirks, and dispensed with micromanagement and red tape.
Early RAND leadership was, above all else, committed to bringing in top talent and jealously guarded the sort of intellectual independence to which their academic hires were accustomed. Taking the mathematics department as an example, RAND hired John Williams, Ted Harris, and Ed Quade to run it. While these were accomplished mathematicians in their own right, these three were also able to attract superlative talents to work under and around them. As Alex Abella writes in Soldiers of Reason, his history of RAND, “No test for ideological correctness was given to join, but then none was needed. The nation’s best and brightest joining RAND knew what they were signing on for, and readily accepted the vision of a rational world — America and its Western allies — engaged in a life-and-death struggle with the forces of darkness: the USSR.”
As the Cold War intensified, the mission became the sell. The aim of RAND, as the historian David Hounshell has it, “was nothing short of the salvation of the human race.”3 The researchers attracted to that project believed that the only environment in which that aim could be realized was independent of the Air Force, its conventional wisdom, and — in particular — its conventional disciplinary boundaries
RAND’s earliest research aligned with the USAF’s (the Army Air Force had become its own service branch in 1947) initial vision: research in the hard sciences to attack problems like satellite launches and nuclear-powered jets.4 However, the mathematician John Davis Williams, Collbohm’s fifth hire, was convinced that RAND needed a wider breadth of disciplines to support the Air Force’s strategic thinking. He made the case to General LeMay, who supervised RAND, that the project needed “every facet of human knowledge to apply to problems.”5 To that end, he argued for recruiting economists, political scientists, and every other kind of social scientist. LeMay, once convinced, implored Williams to hire whoever it took to get the analysis right.
And so they did. RAND’s leadership invested heavily in recruiting the best established and emerging talent in academia. An invitation-only conference organized by Williams in New York in 1947 brought together top political scientists (Bernard Brodie), anthropologists (Margaret Mead), economists (Charles Hitch), sociologists (Hans Speier), and even a screenwriter (Leo Rosten). The promise of influence, exciting interdisciplinary research, and complete intellectual freedom drew many of the attendees to sign up.
Within two years, RAND had assembled 200 of America’s leading academics. The top end of RAND talent was (and would become) full of past (and future) Nobel winners, and Williams worked around many constraints — and eccentricities — to bring them on. For instance, RAND signed a contract with John von Neumann to produce a general theory of war, to be completed during a small slice of his time: that spent shaving. For his shaving thoughts, von Neumann received $200 a month, an average salary at the time.
Beyond the biggest names, RAND was “deliberate, vigorous, and proactive” in recruiting the “first-rate and youthful staff” that made up most of its workforce. The average age of staff in 1950 was under 30.6 Competition between them helped drive the culture of excellence. Essays and working papers were passed around for comments, which were copious — and combative. New ideas had to pass “murder boards.” And the competition spilled into recreational life: Employees held tennis tournaments and boating competitions. James Drake, an aeronautical engineer, invented the sport of windsurfing. The wives of RAND employees — who were, with a few notable exceptions, almost all male — even competed through a cooking club where they tried to make the most "exotic" recipes.
After bringing in such extraordinary talent, RAND’s leadership trusted them to largely self-organize. Department heads were given a budget and were free to spend it as they felt fit. They had control over personnel decisions, which allowed them the flexibility to attract and afford top talent. As a self-styled “university without students,” RAND researchers were affiliated with departments with clear disciplinary boundaries, which facilitated the movement of researchers between RAND and academia. But in practice, both departments and projects were organized along interdisciplinary lines.
The mathematics department brought on an anthropologist. The aeronautics department hired an MD. This hiring strategy paid off in surprising ways. For instance, while modeling the flow of drugs in the bloodstream, a group of mathematicians stumbled upon a technique to solve a certain class of differential equations that came to be used in understanding the trajectory of intercontinental ballistic missiles.
Finding an Institutional Footing
RAND was at the forefront of a postwar explosion in federal funding for science. Hundreds of millions of dollars poured into universities, think tanks, and industrial R&D labs. Almost all of it was directed toward one purpose: maintaining military superiority over the Soviet Union. In 1950, over 90% of the federal research budget came from just two agencies: the Atomic Energy Commission and the Department of Defense.7 Significant portions of this funding went toward basic research with no immediate military applications.8 Vannevar Bush, the influential head of the war-era Office of Scientific Research and Development, had argued for this approach in his 1945 book Science, the Endless Frontier: Freeing up scientists to follow their own research interests would inevitably lead to more innovation and ensure American technological dominance. Bush’s was not the only, or even the dominant, view of how postwar science should be organized — most science funding still went toward applied research — but his views helped inform the organization of a growing number of research institutions.9 No organization embodied this model more than RAND. Air Force contracts were the financial backbone of the organization. They provided the money required to run RAND, while profits were used to fund basic research. In the 1950s, USAF contracts comprised 56% of RAND’s work, while other sponsors made up just 7%.10 That left more than a third of RAND’s capacity open to pursue its own agenda in basic research. Many of the developments made there would be used in their applied research, making it stronger — and more profitable — in the process. This flywheel would become critical to RAND’s success.
Not all of these developments were successful, especially at first. RAND’s early research efforts in systems analysis — an ambitious pursuit in applying mathematical modeling that RANDites were optimistic could produce a holistic “science of warfare” — were flops. The first project, which aimed to optimize a strategic bombing plan on the Soviet Union, used linear programming, state-of-the-art computing, and featured no fewer than 400,000 different configurations of bombs and bombers. It proved of little use to war planners. Its assumptions fell prey to the “specification problem:” trying to optimize one thing, in this case, calculating the most damage for the least cost led to misleading and simplistic conclusions.11
But RAND would soon find its footing, and a follow up to this work became a classic of the age. The 1954 paper Selection and Use of Strategic Air Bases proved the value of RAND’s interdisciplinary approach — though its conclusions were at first controversial. Up to the 1950s, there had been little analysis of how the Strategic Air Command, responsible for the United States’s long range bomber and nuclear deterrent forces, should use its Air Force bases. At the time, the SAC had 32 bases across Europe and Asia. The study, led by political scientist Albert Wohlstetter, found that the SAC was dangerously vulnerable to a surprise Soviet attack. The SAC’s radar defenses wouldn’t be able to detect low-flying Soviet bombers, which could reduce American bombers to ash — and thereby neutralize any threat of retaliation — before the Americans had a chance to react. Wohlstetter’s study recommended that the SAC keep its bombers in the U.S., dispersed at several locations to avoid concentration at any place.
LeMay, RAND’s original benefactor and commander of the SAC, resisted Wohlstetter’s conclusions. He worried the plan would reduce his control over the country’s nuclear fleet: With the SAC based in the U.S., LeMay would have to cede some authority to the rest of the U.S. Air Force. He pushed against it many times, proposing several alternatives in which the SAC kept control over the bombers, but no plan fully addressed the vulnerabilities identified by the report.
Undaunted — and sure of his logic — Wohlstetter pushed his conclusions even further. He proposed a fail-safe mechanism, where nuclear bombers would have to receive confirmation of their attack from multiple checkpoints along the way, to prevent rogue or mistaken orders from being followed. Wohlstetter went around LeMay, to Defense Secretary Charles Wilson and General Nathan Twining, chairman of the Joint Chiefs of Staff, who ultimately accepted the study’s recommendations in full. It took over two decades, but they proved their value in 1980 when a faulty chip erroneously warned of an impending Soviet strike. While no order for a retaliatory attack was issued, had there been one, the fail-safe mechanism would have prevented the bombers from actually attacking the USSR. Selection and Use of Strategic Air Bases was a triumph for RAND. Not only had they provided correct advice to the USAF, they had also proved their independence from the institution’s internal politics.
And the flywheel would prove its value many times over. RAND’s basic research helped drive the development and strategy of ICBMs, the launch of the first meteorological satellite, and, later, on cost reductions in ICBM launch systems.
Diversification and Decline
RAND’s conclusions ran counter to USAF doctrine several times — and each time RAND fought to maintain its independence. When the USAF commissioned RAND to study the Navy’s Polaris program — in order to show that it was inferior to the Air Force’s bombers for nuclear weapon delivery — RAND found that the Polaris missiles were, in fact, superior. The same happened with another study, which challenged the effectiveness of the B-70 bomber in 1959.
Over time, however, these tensions added friction to the relationship. To make matters worse, between 1955 and 1960, the USAF’s budget declined in both absolute terms, and relative to the rest of the defense community. In 1959, the Air Force froze RAND’s budget, presumably due to the budget cuts — and their disputes with RAND.
This situation was not unique to the USAF, or to RAND. As the 1950s rolled into the ’60s, scientists at civilian institutions increasingly moved to disentangle themselves from their military benefactors. Throughout the decade, DOD funding for basic research would only continue to decline.12
RAND weathered the transition by successfully seeking out new customers — the AEC, ARPA, the Office of the Comptroller, the Office of the Assistant Secretary of Defense for International Security Affairs (ISA), NASA, the NSF, the NIH, and the Ford Foundation, to name a few. The percent of the outside funding coming from the USAF dropped from 95% when RAND started to 68% in 1959.13 But their success came at a cost: This diversification is what led to RAND losing its edge in producing the cutting edge of policy and applied science.
Funding diversification reshaped both RAND’s culture and output. The increased number of clients made scheduling researchers’ work harder. Each client expected a different standard of work, and the tolerance levels for RAND’s previously freewheeling style varied. The transaction costs of starting a new contract were much higher and the flexible staffing protocols that had worked for the USAF in the 1950s needed to be systematized. The larger organization led to ballooning internal administration expenses.
Along with all of this, RAND’s increased size attracted more political detractors. In 1958, a RAND paper called Strategic Surrender, which examined the historical conditions for surrender, had generated a political firestorm. Politicians were furious with RAND for exploring conditions under which it would be strategic for the U.S. to surrender. Senators weren’t particularly interested in the study itself, but those who wanted to run for president (like Stuart Symington of Missouri) used it as evidence that the Eisenhower administration was weak on defense.
The Senate even passed a resolution (with an 88–2 margin) prohibiting the use of federal funds for studying U.S. surrender. RAND’s management, realizing that an intentional misinterpretation of their work potentially threatened future funding streams, now had to consider the wider domestic political context of their work. All of these factors changed RAND’s culture from one that encouraged innovation and individuality to one that sapped creativity.
But the biggest change was yet to come. In 1961, Robert McNamara took over the Department of Defense and brought with him a group of RAND scholars, commonly called the “Whiz Kids.” Their most important long-term contribution to U.S. governance was the Planning-Programming-Budgeting System. PPBS took a Randian approach to resource allocation, namely, modeling the most cost-effective ways to achieve desired outcomes. In 1965, after President Johnson faced criticism for poor targeting of his Great Society spending, he required nearly all executive agencies to adopt PPBS. Many RAND alumni were hired by McNamara and his team to help with the Great Society’s budgeting process.
In 1965, Henry Loomis, the deputy commissioner on education, approached RAND about conducting research on teaching techniques. Franklin Collbohm, RAND’s founder and then president, declined. He preferred that RAND stay within the realm of military analysis. RAND’s board disagreed and would eventually push Collbohm out of RAND in 1967. The board thought it was time for a change in leadership — and to RAND’s nonmilitary portfolio.
The entry of a new president, Henry S. Rowen, an economist who had started his career at RAND, cemented this change. By 1972, the last year of Rowen’s tenure, almost half of all RAND projects were related to social science. For better or worse, this eroded RAND’s ability to take on cutting-edge scientific research and development.
RAND entered domestic policy research with a splash — or, rather, a belly flop. The politics of social policy research were markedly different from working with the DOD. For one, there were substantially more stakeholders — and they were more vocal about voicing their disagreements. One crucial example is when RAND proposed police reforms in New York City, but pressure from the police unions forced them to retract.
John Lindsay, the Republican mayor of New York, had tasked RAND with improving the New York Police Department, which had recently been implicated in narcotics scams, corruption, and police brutality. The report showed that in less than 5% of the cases in which an officer was charged with a crime or abusing a citizen did the officers receive anything more than a reprimand. The findings were leaked to The New York Times, which added to the impression among the police that RAND was the mayor’s mouthpiece.
RAND, for the first time, had to face the reality of local politics: a sometimes hostile environment, multiple stakeholders who sometimes acted in bad faith, and none of the free reign that characterized their first decades. RAND’s experience with the police report, and the controversy over the study of surrender, led RAND to be more conservative about the research it put out. And additionally, the focus on policy research crowded out the scientific research.
For example, beginning in the 1970s, RAND’s applied mathematics research output slowed to a trickle, before stopping altogether in the 1990s. It was replaced by mathematics education policy. The same is true for physics, chemistry, and astronomy. Another emblematic development in the dilution of RAND’s focus was the founding in 1970 of the Pardee RAND Graduate School, the nation’s first Ph.D.-granting program in policy analysis. While the idea of training the next generation in RAND techniques is admirable, RAND in the early years explicitly defined itself as a “university without students.”
RAND is still an impressive organization. It continues to produce successful policy research, which commands the eyes of policymakers in over 82 federal organizations and across dozens of local and even foreign governments. Still, their work today is inarguably less groundbreaking and innovative than it was in the ’50s. This relative decline was partially caused by internal policy choices, and partially by the eventual loss of their initial team of leading scientists. But part of it was also inevitable: We no longer live in an era when branches of the U.S. military can cut massive blank checks to think tanks in the interest of beating the Soviets. The successes of 1950s RAND do come with lessons for modern research organizations — about the importance of talent, the relevance of institutional culture, and the possibilities of intellectual freedom — but the particular conditions that created them can’t be replicated. It is remarkable that they existed at all.
Modern Magic at RAND
The following commentary comes directly from RAND’s CEO, Jason Matheny.
RAND CEO Jason Matheny here. Your readers may recall from my appearance on your podcast last year that I, too, am a RAND history nerd. There are many great details in your Asterisk article about RAND’s early contributions in the 1950s and ‘60s. Thanks for bringing them to life.
RAND’s contributions in the last five decades have been no less consequential. The world’s challenges are certainly different from the ones RAND researchers confronted in the early years. But it is RAND’s ability to reorient itself toward the biggest challenges that has been our “magic.” We shouldn’t expect or want RAND to look the same as it did during the Cold War.
I thought your readers would be interested to pick up where your story stops. And since your article focuses on national security, I’ll concentrate my comments there. (That said, there have been just as many breakthroughs in RAND’s social and economic policy analyses over the years.)
RAND’s security research in the modern era has been forward-looking, has challenged long-held wisdom, and has anticipated once-unthinkable threats. And I’m not saying this only as RAND’s CEO. Before I joined RAND two years ago, I was one of countless people at the White House and elsewhere in government who relied on RAND analysis to make critical decisions.
Many recent RAND studies will remain classified for years. While their full impact will be assessed with time — much as was the case with RAND’s work in the 1950s and 1960s – they have been among RAND’s most influential. Below are some examples of projects that we can describe here:
China: RAND was among the first research organizations to systematically analyze China’s military buildup and make direct comparisons to the U.S. military.
Russia: RAND was among the first organizations to identify Russia’s growing military capabilities following its 2008 war in Georgia and the threat these posed to new NATO members in the Baltic states. This work prompted important planning and infrastructure changes that are being used today to support Ukraine.
U.S. military power: RAND’s series of overmatch studies transformed policymakers’ understanding the loss of U.S. military superiority in key areas over time.
Operating in the Pacific theater: RAND was among the first to highlight the vulnerability of the U.S. military’s forward infrastructure in the Pacific and ways to overcome that vulnerability.
Nuclear strategy: RAND’s recent work on nuclear deterrence, including wargames analyzing nuclear-armed regional adversaries, brought about a resurgence of deterrence thinking within the government.
B-21: RAND analysis of penetrating versus standoff bomber capabilities led directly to the decision to establish the B-21 program.
Military forces: RAND‘s work on military personnel, the ability to develop and sustain the all-volunteer force over time, appropriate pay and benefits for the force, and the vulnerabilities to service members and their families, has been the primary source of analysis for decisionmakers within the Department of Defense and Congress.
Drones: RAND’s analysis of small UAVs and swarming options was the first to analyze how a sensor grid can substantially strengthen deterrence in the Asia-Pacific region. Current DoD programs can be traced directly to this pathbreaking analysis.
PTSD and TBI: RAND’s work on the invisible wounds of war, PTSD, and traumatic brain injury, was the first careful documentation of psychological and cognitive injuries from modern combat. This work launched a society-wide effort to detect and treat such injuries.
Logistics: RAND analysis prompted the revolution in combat logistics in both the Air Force and the Army, emphasizing wartime flexibility and resilience as the organizing principles for supply and maintenance.
AI: RAND was early in systematically evaluating how defense organizations could integrate contemporary AI methods based on deep learning, in evaluating large language models, and in assessing threats to model security.
With rapid developments in emerging technology and an increasingly confrontational PRC government, the world needs RAND’s analysis more than ever. I know that your readers care deeply about these challenges. Those who want to work toward solutions should consider working at RAND or applying to our new master's degree program in national security policy.
To hear more from Jason, check out the two-hour interview we did last year on ChinaTalk, which was my favorite episode of 2023.
Transcript:
Click to open in Apple Podcasts or Spotify.
According to the University of Pennsylvania’s Global Go To Think Tank Index Report.
Similar stories of outsiders applying quantitative thinking improving performance also played out in other branches. Such experiences were also seen in the Navy, where better usage of anti-submarine depth charges led to higher efficiency to the extent that German naval planners thought that the U.S. had invented a new type of depth charge.
D. Hounshell, “The Cold War, RAND, and the Generation of Knowledge, 1946–1962,” Historical Studies in the Physical and Biological Sciences 27, no. 2(1997): 237–267.
Unfortunately, the researchers at RAND couldn’t figure out how to make a nuclear plane that didn’t irradiate the pilots.
Sharon Ghamari-Tabrizi, The Worlds of Herman Kahn: The Intuitive Science of Thermonuclear War (Cambridge: Harvard University Press, 2005), 52.
Abella, Soldiers of Reason.
Dan Kevles, “Cold War and Hot Physics: Science, Security, and the American State, 1945–1956,” Historical Studies in the Physical and Biological Sciences 20, no. 2 (1990): 244.
Audra J. Wolfe, Competing with the Soviets: Science, Technology, and the State in Cold War America (Baltimore: The Johns Hopkins University Press, 2013), 36.
Ibid. 37–42.
Bruce L.R. Smith, The Rand Corporation: Case Study of a Nonprofit Advisory Corporation (Cambridge: Harvard University Press, 1966), 167.
D. Hounshell, “The Cold War, RAND, and the Generation of Knowledge, 1946–1962,” Historical Studies in the Physical and Biological Sciences 27, no. 2(1997): 237–267.
Wolfe, Competing with the Soviets, 134–138, 145.
Bruce L. R. Smith, The RAND Corporation: Case Study of a Nonprofit Advisory Corporation (Cambridge: Harvard University Press, 1966).
 | A guest post by
|
Strange that no mention is made of RAND's association with the Pentagon Papers. I was a RAND analyst for 5 years, and the first scholar brought into that study. Dan Ellsberg and others followed. While still at RAND, I joined with Ellsberg and others in a letter to the Times and WaPo urging immediate US withdrawal from Vietnam. And I testified for Ellsberg at his trial in Los Angeles. Needless to say, all this activism did not sit well with RAND's leadership (Rowen), which pressured some of us to resign. Not a sparkling moment for the organization.
A bit rich of the CEO of RAND to say how they alerted the world to Russia's aggression, when they have as their Russia Chair Sam Charap, the US individual outside government who has probably had more influence in pushing Ukraine towards defeat and subjugation by Russia than anyone.