Who Else Wants to Get Rich?

November 13, 2022

There are many ways to get rich.  One of the best ways is through long-term investing.

A wise long-term investment for most investors is an S&P 500 index fund.  It’s just simple arithmetic, as Warren Buffett and Jack Bogle frequently observe: https://boolefund.com/warren-buffett-jack-bogle/

But you can do significantly better—roughly 18% per year (instead of 10% per year)—by systematically investing in cheap, solid microcap stocks.

Most professional investors never consider microcaps because their assets under management are too large.  Microcaps aren’t as profitable for them.  That’s why there continues to be a compelling opportunity for savvy investors.  Because microcaps are largely ignored, many get quite cheap on occasion.

Warren Buffett earned the highest returns of his career when he could invest in microcap stocks.  Buffett says he’d do the same today if he were managing small sums: https://boolefund.com/buffetts-best-microcap-cigar-butts/

Look at this summary of the CRSP Decile-Based Size and Return Data from 1927 to 2020:

Decile Market Cap-Weighted Returns Equal Weighted Returns Number of Firms (year-end 2020) Mean Firm Size (in millions)
1 9.67% 9.47% 179 145,103
2 10.68% 10.63% 173 25,405
3 11.38% 11.17% 187 12,600
4 11.53% 11.29% 203 6,807
5 12.12% 12.03% 217 4,199
6 11.75% 11.60% 255 2,771
7 12.01% 11.99% 297 1,706
8 12.03% 12.33% 387 888
9 11.55% 12.51% 471 417
10 12.41% 17.27% 1,023 99
9+10 11.71% 15.77% 1,494 199

(CRSP is the Center for Research in Security Prices at the University of Chicago.  You can find the data for various deciles here:  http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html)

The smallest two deciles—9+10—comprise microcap stocks, which typically are stocks with market caps below $500 million.  What stands out is the equal weighted returns of the 9th and 10th size deciles from 1927 to 2020:

Microcap equal weighted returns = 15.8% per year

Large-cap equal weighted returns = ~10% per year

In practice, the annual returns from microcap stocks will be 1-2% lower because of the difficulty (due to illiquidity) of entering and exiting positions.  So we should say that an equal weighted microcap approach has returned 14% per year from 1927 to 2020, versus 10% per year for an equal weighted large-cap approach.

 

VALUE SCREEN: +2-3%

By systematically implementing a value screen—e.g., low EV/EBITDA or low P/E—to a microcap strategy, you can add 2-3% per year.

 

IMPROVING FUNDAMENTALS: +2-3%

You can further boost performance by screening for improving fundamentals.  One excellent way to do this is using the Piotroski F_Score, which works best for cheap micro caps.  See:  https://boolefund.com/joseph-piotroski-value-investing/

 

BOTTOM LINE

If you invest in microcap stocks, you can get about 14% a year.  If you also use a simple screen for value, that adds at least 2% a year.  If, in addition, you screen for improving fundamentals, that adds at least another 2% a year.  So that takes you to 18% a year, which compares quite well to the 10% a year you could get from an S&P 500 index fund.

What’s the difference between 18% a year and 10% a year?  If you invest $50,000 at 10% a year for 30 years, you end up with $872,000, which is good.  If you invest $50,000 at 18% a year for 30 years, you end up with $7.17 million, which is much better.

Please contact me if you would like to learn more.

    • My email: jb@boolefund.com.
    • My cell: 206.518.2519

 

BOOLE MICROCAP FUND

An equal weighted group of micro caps generally far outperforms an equal weighted (or cap-weighted) group of larger stocks over time.  See the historical chart here:  https://boolefund.com/best-performers-microcap-stocks/

This outperformance increases significantly by focusing on cheap micro caps.  Performance can be further boosted by isolating cheap microcap companies that show improving fundamentals.  We rank microcap stocks based on these and similar criteria.

There are roughly 10-20 positions in the portfolio.  The size of each position is determined by its rank.  Typically the largest position is 15-20% (at cost), while the average position is 8-10% (at cost).  Positions are held for 3 to 5 years unless a stock approaches intrinsic value sooner or an error has been discovered.

The mission of the Boole Fund is to outperform the S&P 500 Index by at least 5% per year (net of fees) over 5-year periods.  We also aim to outpace the Russell Microcap Index by at least 2% per year (net).  The Boole Fund has low fees.

 

 

 

Disclosures: Past performance is not a guarantee or a reliable indicator of future results. All investments contain risk and may lose value. This material is distributed for informational purposes only. Forecasts, estimates, and certain information contained herein should not be considered as investment advice or a recommendation of any particular security, strategy or investment product. Information contained herein has been obtained from sources believed to be reliable, but not guaranteed. No part of this article may be reproduced in any form, or referred to in any other publication, without express written permission of Boole Capital, LLC.

The Success Equation: Untangling Skill and Luck

November 6, 2022

Michael Mauboussin wrote a great book called The Success Equation: Untangling Skill and Luck in Business, Sports, and Investing (Harvard Business Press, 2012).

Here’s an outline for this blog post:

    • Understand were you are on the skill-luck continuum
    • Assess sample size, significance, and swans
    • Always consider a null hypothesis
    • Think carefully about feedback and rewards
    • Make use of counterfactuals
    • Develop aids to guide and improve your skill
    • Have a plan for strategic interactions
    • Make reversion to the mean work for you
    • Know your limitations

 

UNDERSTAND WHERE YOU ARE ON THE SKILL-LUCK CONTINUUM

If an activity is mostly skill or mostly luck, it’s generally easy to classify it as such.  But many activities are somewhere in-between the two extremes, and it’s often hard to say where it falls on the continuum between pure skill and pure luck.

An activity dominated by skill means that results can be predicted reasonably well.  (You do need to consider the rate at which skill changes, though.)  A useful statistic is one that is persistent—the current outcome is highly correlated with the previous outcome.

An activity dominated by luck means you need a very large sample to detect the influence of skill.  The current outcome is not correlated with the previous outcome.

Obviously the location of an activity on the continuum gives us guidance on how much reversion to the mean is needed in making a prediction.  In an activity that is mostly skill, the best estimate for the next outcome is the current outcome.  In an activity that is mostly luck, the best guess for the next outcome is close to the base rate (the long-term average), i.e., nearly a full reversion to the mean.

Our minds by nature usually fail to regress to the mean as much as we should.  That’s because System 1—the automatic, intuitive part of our brain—invents coherent stories based on causality.  This worked fine during most of our evolutionary history.  But when luck plays a significant role, there has to be substantial reversion to the mean when predicting the next outcome.

 

ASSESS SAMPLE SIZE, SIGNIFICANCE, AND SWANS

Even trained scientists have a tendency to believe that a small sample of a population is representative of the whole population.  But a small sample can deviate meaningfully from the larger population.

If an activity is mostly skill, then a small sample will be representative of the larger population from which it is drawn.  If an activity is mostly luck, then a small sample can be significantly different from the larger population.  A small sample is not reliable when an activity is mostly luck—we need a large sample in this case in order to glean information.

In business, it would be an error to create a sample of all the companies that used a risky strategy and won, without also taking into account all the companies that used the same strategy and lost.  A narrow sample of just the winners would obviously be a biased view of the strategy’s quality.

Also be careful not to confuse statistical significance with economic significance.  Mauboussin quotes Deirdre McCloskey and Stephen Ziliak: “Tell me the oomph of your coefficient; and do not confuse it with mere statistical significance.”

Lastly, it’s important to keep in mind that some business strategies can produce a long series of small gains, followed by a huge loss.  Most of the large U.S. banks pursued such a strategy from 2003-2007.  It would obviously be a big mistake to conclude that a long series of small gains is safe if in reality it is not.

Another example of ignoring black swans is Long-Term Capital Management.  The fund’s actual trades were making about 1% per year.  But LTCM argued that these trades had infintessimally small risk, and so they levered the trades at approximately 40:1.  Many banks didn’t charge LTCM anything for the loan because LTCM was so highly regarded at the time, having a couple of Nobel Prize winners, etc.  Then a black swan arrived—the Asian financial crisis in 1998.  LTCM’s trades went against them, and because of the astronomically high leverage, the fund imploded.

 

ALWAYS CONSIDER A NULL HYPOTHESIS

Always compare the outcomes to what would have been generated under the null hypothesis.  Many streaks can easily be explained by luck alone.

Mauboussin gives the example of various streaks of funds beating the market.  Andrew Mauboussin and Sam Arbesman did a study on this.  They assumed that the probability a given fund would beat the S&P 500 Index was equal to the fraction of active funds that beat the index during a given year.  For example, 52 percent of funds beat the S&P 500 in 1993, so the null model assigns the same percentage probability that any given fund would beat the market in that year.  Mauboussin and Arbesman then ran ten thousand random simulations.

They determined that, under the null model—pure luck and no skill—146.9 funds would have a 5-year market-beating streak, 53.6 funds would have a 6-year streak, 21.4 funds would have a 7-year streak, 7.6 funds would have an 8-year streak, and 3.0 funds would have a 9-year streak.  They compared these figures to the actual empirical frequencies:  206 funds had 5-year streaks, 119 had 6-year streaks, 75 had 7-year streaks, 23 had 8-year streaks, and 28 had 9-year streaks.

So there were many more streaks in the empirical data than the null model generated.  This meant that some of those streaks involved the existence of skill.

 

THINK CAREFULLY ABOUT FEEDBACK AND REWARDS

Everybody wants to improve.  The keys to improving performance include high-quality feedback and proper rewards.

Only a small percentage of people achieve expertise through deliberate practice.  Most people hit a performance plateau and are satisfied to stay there.  Of course, for many activities—like driving—that’s perfectly fine.

The deliberate practice required to develop true expertise involves a great deal of hard and tedious work.  It is not pleasant.  It requires thousands of hours of very focused effort.  And there must be a lot of timely and accurate feedback in order for someone to keep improving and eventually attain expertise.

Even if you’re not pursuing expertise, the keys to improvement are still focused practice and high-quality feedback.

In activities where skill plays a significant role, actual performance is a reasonable measure of progress.  Where luck plays a strong role, the focus must be on the process.  Over shorter periods of time—more specifically, over a relatively small number of trials—a good process can lead to bad outcomes, and a bad process can lead to good outcomes.  But over time, with a large number of trials, a good process will yield good outcomes overall.

The investment industry struggles in this area.  When a strategy does well over a short period of time, quite often it is marketed and new investors flood in.  When a strategy does poorly over a short period of time, very often investors leave.  Most of the time, these strategies mean revert, so that the funds that just did well do poorly and the funds that just did poorly do well.

Another area that’s gone off-track is rewards for executives.  Stock options have become a primary means of rewarding executives.  But the payoff from a stock option involves a huge amount of randomness.  In the decade of the 1990’s, even poor-performing companies saw their stocks increase a great deal.  In the decade of the 2000’s, many high-performing companies saw their stocks stay relatively flat.  So stock options on the whole have not distinguished between skill and luck.

A solution would involve having the stock be measured relative to an index or relative to an appropriate peer group.  Also, the payoff from options could happen over longer periods of time.

Lastly, although executives—like the CEO—are much more skillful than their junior colleagues, often executive success depends to a large extent on luck while the success of those lower down can be attributed almost entirely to skill.  For instance, the overall success of a company may only have a 0.3 correlation with the skill of the CEO.  And yet the CEO would be paid as if the company’s success was highly correlated with his or her skill.

 

MAKE USE OF COUNTERFACTUALS

Once we know what happened in history, hindsight bias naturally overcomes us and we forget how unpredictable the world looked beforehand.  We come up with reasons to explain past outcomes.  The reasons we invent typically make it seem as if the outcomes were inevitable when they may have been anything but.

Mauboussin says a good way to avoid hindsight bias is to engage in counterfactual thinking—a careful consideration of what could have happened but didn’t.

Mauboussin gives an example in Chapter 6 of the book: MusicLab.  Fourteen thousand people were randomly divided into 8 groups—each 10% of the total number of people—and one independent group—20% of the total number of people.  There were forty-eight songs from unknown bands.  In the independent group, each person could listen to each song and then decide to download it based on that alone.  In the other 8 groups, for each song, a person would see how many other people in his or her group had already downloaded the song.

You could get a reasonable estimate for the “objective quality” of a song by looking at how the independent group rated them.

But in the 8 “social influence” groups, strange things happened based purely on luck—or which songs were downloaded early on and which were not.  For instance, a song “Lockdown” was rated twenty-sixth in the independent group.  But it was the number-one hit in one of the social influence worlds and number forty in another.

In brief, to maintain an open mind about the future, it is very helpful to maintain an open mind about the past.  We have to work hard to overcome our natural tendency to view what happened as having been inevitable.  System 1 always creates a story based on causality—System 1 wants to explain simply what happened and close the case.

If we do the Rain Dance and it rains, then to the human brain, it looks like the dance caused the rain.

But when we engage System 2 (the logical, mathematical part of our brain)—which requires conscious effort—we can come to realize that the Rain Dance didn’t cause the rain.

 

DEVELOP AIDS TO GUIDE AND IMPROVE YOUR SKILL

Depending on where an activity lies on the pure luck to pure skill continuum, there are different ways to improve skill.

When luck predominates, to improve our skill we have to focus on learning the process for making good decisions.  A good process must be well grounded in three areas:

  • analytical
  • psychological
  • organizational

In picking an undervalued stock, the analytical part means finding a discrepancy between price and value.

The psychological part of a good process entails an identification of the chief cognitive biases, and techniques to mitigate the influence of these cognitive biases.  For example, we all tend to be wildly overconfident when we make predictions.  System 1 automatically makes predictions all the time.  Usually this is fine.  But when the prediction involves a probabilistic area of life—such as an economy, a stock market, or a political situation—System 1 makes errors systematically.  In these cases, it is essential to engage System 2 in careful statistical thinking.

The organizational part of a good process should align the interests of principals and agents—for instance, shareholders (principals) and executives (agents).  If the executives own a large chunk of stock, then their interests are much more aligned with shareholder interests.

Now consider the middle of the continuum between luck and skill.  In this area, a checklist can be very useful.  A doctor caring for a patient is focused on the primary problem and can easily forget about the simple steps required to minimize infections.  Following the suggestion of Peter Pronovost, many hospitals have introduced simple checklists.  Thousands of lives and hundreds of millions of dollars have been saved, as the checklists have significantly reduced infections and deaths related to infections.

A checklist can also help in a stressful situation.  The chemicals of stress disrupt the functioning of the frontal lobes—the seat of reason.  So a READ-DO checklist gets you to take the concrete, important steps even when you’re not thinking clearly.

Writes Mauboussin:

Checklists have never been shown to hurt performance in any field, and they have helped results in a great many instances.

Finally, anyone serious about improving their performance should write down—if possible—the basis for every decision and then measure honestly how each decision turned out.  This careful measurement is the foundation for continual improvement.

The last category involves activities that are mostly skill.  The key to improvement is deliberate practice and good feedback.  A good coach can be a great help.

Even experts benefit from a good coach.  Feedback is the single most powerful way to improve skill.  Being open to honest feedback is difficult because it means being willing to admit where we need to change.

Mauboussin concludes:

One simple and inexpensive technique for getting feedback is to keep a journal that tracks your decisions.  Whenever you make a decision, write down what you decided, how you came to that decision, and what you expect to happen.  Then, when the results of that decision are clear, write them down and compare them with what you thought would happen.  The journal won’t lie.  You’ll see when you’re wrong.  Change your behavior accordingly.

 

HAVE A PLAN FOR STRATEGIC INTERACTIONS

The weaker side won more conflicts in the twentieth century than in the nineteenth.  This is because the underdogs learned not to go toe-to-toe with a stronger foe.  Instead, the underdogs pursued alternative tactics, like guerrilla warfare.  If you’re an underdog, complicate the game by injecting more luck.

Initially weaker companies almost never succeed by taking on established companies in their core markets.  But, by pursuing disruptive innovation—as described by Professor Clayton Christensen—weaker companies can overcome stronger companies.  The weaker companies pursue what is initially a low-margin part of the market.  The stronger companies have no incentive to invest in low-margin innovation when they have healthy margins in more established areas.  But over time, the low-margin technology improves to the point where demand for it increases and profit margins typically follow.  By then, the younger companies are already ahead by a few of years, and the more established companies usually are unable to catch up.

 

MAKE REVERSION TO THE MEAN WORK FOR YOU

Mauboussin writes:

We are all in the business of forecasting.

Reversion to the mean is difficult for our brains to understand.  As noted, System 1 always invents a cause for everything that happens.  But often there is no specific cause.

Mauboussin cites an example given by Daniel Kahneman: Julie is a senior in college who read fluently when she was four years old.  Estimate her GPA.

People often guess a GPA of around 3.7.  Most people assume that being precocious is correlated with doing well in college.  But it turns out that reading at a young age is not related to doing well in college.  That means the best guess for the GPA would be much closer to the average.

Mauboussin adds:

Reversion to the mean is most pronounced at the extremes, so the first lesson is to recognize that when you see extremely good or bad results, they are unlikely to continue that way.  This doesn’t mean that good results will necessarily be followed by bad results, or vice versa, but rather that the next thing that happens will probably be closer to the average of all things that happen.

 

KNOW YOUR LIMITATIONS

There is always a great deal that we simply don’t know and can’t know.  We must develop and maintain a healthy sense of humility.

Predictions are often difficult in many situations.  The sample size and the length of time over which you measure are essential.  And you need valid data.

Moreover, things can change.  If fiscal policy has become much more stimulative than it used to be, then bear markets may—or may not—be shallower and shorter.  And stocks may generally be higher than previously, as Ben Graham pointed out in a 1963 lecture, “Securities in an Insecure World”: http://jasonzweig.com/wp-content/uploads/2015/04/BG-speech-SF-1963.pdf

If monetary policy is much more stimulative than before—including a great deal of money-printing and zero or negative interest rates—then the long-term average of stock prices could conceivably make another jump higher.

The two fundamental changes just mentioned are part of why most great value investors never try to time the market.  As Buffett has said:

  • Forecasts may tell you a great deal about the forecaster;  they tell you nothing about the future.
  • I make no effort to predict the course of general business or the stock market.  Period.
  • I don’t invest a dime based on macro forecasts.

Henry Singleton—who has one of the best capital allocation records of all time—perhaps put it best:

I don’t believe all this nonsense about market timing.  Just buy very good value and when the market is ready that value will be recognized.

 

BOOLE MICROCAP FUND

An equal weighted group of micro caps generally far outperforms an equal weighted (or cap-weighted) group of larger stocks over time.  See the historical chart here:  https://boolefund.com/best-performers-microcap-stocks/

This outperformance increases significantly by focusing on cheap micro caps.  Performance can be further boosted by isolating cheap microcap companies that show improving fundamentals.  We rank microcap stocks based on these and similar criteria.

There are roughly 10-20 positions in the portfolio.  The size of each position is determined by its rank.  Typically the largest position is 15-20% (at cost), while the average position is 8-10% (at cost).  Positions are held for 3 to 5 years unless a stock approaches intrinsic value sooner or an error has been discovered.

The mission of the Boole Fund is to outperform the S&P 500 Index by at least 5% per year (net of fees) over 5-year periods.  We also aim to outpace the Russell Microcap Index by at least 2% per year (net).  The Boole Fund has low fees.

If you are interested in finding out more, please e-mail me or leave a comment.

My e-mail: jb@boolefund.com

Disclosures: Past performance is not a guarantee or a reliable indicator of future results. All investments contain risk and may lose value. This material is distributed for informational purposes only. Forecasts, estimates, and certain information contained herein should not be considered as investment advice or a recommendation of any particular security, strategy or investment product. Information contained herein has been obtained from sources believed to be reliable, but not guaranteed. No part of this article may be reproduced in any form, or referred to in any other publication, without express written permission of Boole Capital, LLC.

Bad Blood

October 30, 2022

Bad Blood: Secrets and Lies in a Silicon Valley Startup, by John Carreyrou, is hands-down one of the best business books I’ve ever read.  (It’s up there with Business Adventures, by John Brooks, and Shoe Dog, by Phil Knight.)  The book tells the story of the rise and fall of Theranos. 

In brief, here’s why it’s such a great book: Carreyrou was the investigative reporter who broke the Theranos story in 2015.  Carreyrou interviewed 150 people and withstood enormous pressure from the company’s charismatic CEO and her attorneys—led by one of the best and most feared lawyers in the country.  Other key whistle-blowers also withstood great pressure.  Many of the facts of the story are unbelievable.  And finally, Carreyrou does an outstanding job telling the story.

Theranos was a company founded by Stanford dropout Elizabeth Holmes when she was just 19 years old.  She claimed that she had invented a semi-portable device that could do every kind of blood test using only a tiny drop of blood obtained by a pin prick to the end of a finger.  Holmes declared that you could get the results in a few hours and at a much lower price than what other labs charged.

Had Theranos’s technology worked, it would have been revolutionary.  The only problem: it never came close to working.  There are different kinds of blood tests that require different laboratory instruments.  If you do one kind of blood test on a tiny sample, there isn’t enough blood left to do the other kinds of blood tests.

But Holmes believed in her vision so much, and she was such a charismatic and brilliant saleswoman, that she raised close to $1 billion dollars from investors.  This included $700 million in late 2013.  How?  See this CNN Business interview (Sept. 20, 2018): https://www.youtube.com/watch?v=BXfw-S62ISE

As Carreyrou explains to Julia Chatterley in the CNN Business interview: Holmes launched the purportedly innovative technology in Walgreens stores in California and Arizona.  In addition, when seeking investors, she focused on the family offices of billionaires while avoiding investors with any sophistication related to next-gen diagnostics.  When the less sophisticated investors learned that Theranos was giving blood tests in Walgreens stores, they assumed that its technology must be real.

Theranos offered 250 blood tests.  Unbeknownst to investors and to Theranos’s business partners—Walgreens and Safeway—240 of these tests were actually done on Siemens machines.  Theranos would collect tiny blood samples from finger pricks and then dilute them so that the Siemens machine could analyze them.  (This also required modifications to the Siemens machines that Theranos engineers had figured out.)

Only 10 of the 250 blood tests offered by Theranos were done on the company’s machine, the Edison.  Moreover, the Edison was shockingly unreliable.  Ultimately, a million blood tests done by Edison machines were voided.

Here is the outline for this blog post.  (The outline does not include the Prologue and the Epilogue, which I also touch on.)

    1. A Purposeful Life
    2. The Gluebot
    3. Apple Envy
    4. Goodbye East Paly
    5. The Childhood Neighbor
    6. Sunny
    7. Dr. J
    8. The miniLab
    9. The Wellness Play
    10. “Who is LTC Shoemaker?”
    11. Lighting a Fuisz
    12. Ian Gibbons
    13. Chiat/Day
    14. Going Live
    15. Unicorn
    16. The Grandson
    17. Fame
    18. The Hippocratic Oath
    19. The Tip
    20. The Ambush
    21. Trade Secrets
    22. La Mattanza
    23. Damage Control
    24. The Empress Has No Clothes

 

PROLOGUE

November 17, 2006.  Henry Mosley, Theranos’s chief financial officer.  Good news from Tim Kemp: Elizabeth Holmes, the twenty-two year old founder, had shown off the company’s system to executives at Novartis, a European drug giant. She had told Kemp, “It was perfect!”

Carreyrou writes:

This was a pivotal moment for Theranos.  The three-year-old startup had progressed from an ambitious idea Holmes had dreamed up in her Stanford dorm room to an actual product a huge multinational corporation was interested in using.

Mosley was a veteran of Silicon Valley.  Carreyrou again:

What had drawn Mosley to Theranos was the talent and experience gathered around Elizabeth.  She might be young, but she was surrounded by an all-star cast.  The chairman of her board was Donald L. Lucas, the venture capitalist who had groomed billionaire software entrepreneur Larry Ellison and helped him take Oracle Corporation public in the mid-1980s. Lucas and Ellison had both put some of their own money into Theranos.

Another board member with a sterling reputation was Channing Robertson, the associate dean of Stanford’s School of Engineering. Robertson was one of the stars of the Stanford faculty… Based on the few interactions Mosley had had with him, it was clear Robertson thought the world of Elizabeth.

Carreyrou says that Theranos also had a strong management team, which impressed Mosley.  Furthermore, Mosley understood that the market Theranos was targeting was huge.  Pharmaceutical companies spent tens of billions of dollars on clinical trials to test new drugs each year.  If Theranos could make itself indispensable to them and capture a fraction of that spending, it could make a killing.

Carreyrou notes that Elizabeth had asked Mosley for some financial projections, which he provided. Elizabeth asked him to revise the projections upward.  Mosley was a bit uncomfortable, but he knew it was a part of the game new tech startups played to attract VC money.

Something wasn’t right, though.  Although Elizabeth was enthusiastic about the presentation to Novartis, some of her colleagues seemed downcast.  With some digging, Mosley found out the problem from Shaunak, a fellow employee. Theranos’s blood-testing system was unreliable. This was the first Mosley heard about the issue.

Well, there was a reason it always seemed to work, Shaunak said.  The image on the computer screen showing the blood and settling into he little wells was real.  But you never knew whether you were going to get a result or not.  So they’d recorded a result from one of the times it worked. It was that recorded result that was displayed at the end of each demo.

Mosley was stunned.  He thought the result were extracted in real time from the blood inside the cartridge.  That was certainly what the investors he brought by were led to believe.  What Shaunak had just described sounded like a sham. It was OK to be optimistic and aspirational when you pitched investors, but there was a line not to cross.  And this, in Mosley’s view, crossed.

Later, Mosley politely raised the issue with Elizabeth.  Mosley said they couldn’t keep fooling investors.  Elizabeth’s expression immediately became hostile.  She told Mosley he wasn’t a team player and said he should leave immediately.  Elizabeth had fired him.

 

A PURPOSEFUL LIFE

Elizabeth wanted to be an entrepreneur at a young age.  Carreyrou:

When she was seven, she set out to design a time machine and filled up a notebook with detailed engineering drawings.

When she was nine or ten, one of her relatives asked her at a family gathering the question every boy and girl is asked sooner or later: “What do you want to do when you grow up?”

Without skipping a beat, Elizabeth replied, “I want to be a billionaire.”

These weren’t the idle words of a child.  Elizabeth uttered them with the utmost seriousness and determination, according to a family member who witnessed the scene.

Carreyrou explains that Elizabeth’s parents encouraged her ambition based on a distinguished family history.  She descended from Charles Louis Fleischmann on her father’s side.  Fleischmann was a Hungarian immigrant who created the Fleischmann Yeast Company, which was remarkably successful.  The Fleischmanns were one of the wealthiest families in America at the beginning of the twentieth century. 

Bettie Fleischmann, Charles’s daughter, married her father’s Danish physician, Dr. Christian Holmes.  He was Elizabeth’s great-great-grandfather.

Aided by the political and business connections of his wife’s wealthy family, Dr. Holmes established Cincinnati General Hospital and the University of Cincinnati’s medical school.  So the case could be made—and it would in fact be made to the venture capitalists clustered on Sand Hill Road near the Stanford University campus—that Elizabeth didn’t just inherit entrepreneurial genes, but medical ones too.

Elizabeth’s mother, Noel, also had notable family background.  Noel’s father was a West Point graduate who became a high-ranking Pentagon official who oversaw the shift from draft-based military to an all-volunteer force in the early 1970s.  The Daoust line could be traced back to one of Napoleon’s top field generals.

Chris Holmes also made sure to teach his daughter about the failings of his father and grandfather.  They had cycled through marriages and struggled with alcoholism while they squandered the family fortune.  Elizabeth later explained that she learned about greatness, but also about what happens if you don’t have a high purpose—your character and qualify of life suffered.

When Elizabeth was a kid, she liked to play monopoly with her cousins and brother.  She was intensely competitive.  She got furious when she lost and at least a few times ran right through a screen on the front door of the condo owned by her aunt and uncle.

Elizabeth became a straight-A student in high school by working hard and sacrificing sleep.  Stanford was the natural choice for someone interested in computers and science who wanted to be an entrepreneur.  She was accepted to Stanford as a President’s Scholar in the spring of 2002.

Carreyrou explains how Elizabeth got interested in biotechnology:

Her father had drilled into her the notion that she should live a purposeful life.  During his career in public service, Chris Holmes had overseen humanitarian efforts like the 1980 Mariel boatlift, in which more than one hundred thousand Cubans and Haitians migrated to the United States.  There were pictures around the house of him provided disaster relief in war-torn countries.  The message Elizabeth took away from them is that if she wanted to truly leave her mark on the world, she would need to accomplish something that furthered the greater good, not just become rich.  Biotechnology offered the prospect of achieving both.  She chose to study chemical engineering, a field that provided a natural gateway to the industry.

Elizabeth took Introduction to Chemical Engineering from star faculty member Channing Robertson.  She also convinced him to let her work in his lab.  She had a boyfriend for a time, but broke up telling him that she was starting a company and wouldn’t have any time to date.

After her freshman year, Elizabeth had a summer internship at the Genome Institute of Singapore.  Earlier that year (2003), severe acute respiratory syndrome (SARS) had hit Asia.  Elizabeth’s job was testing patient specimens gathered using low-tech methods like syringes.  She thought there was a better way.  Carreyrou:

When she got back home to Houston, she sat down at her computer for five straight days, sleeping one or two hours a night and eating from trays of food her mother brought her.  Drawing from new technologies she had learned about during her internship and in Robertson’s classes, she wrote a patent application for an arm patch that would simultaneously diagnose medical conditions and treat them.

Robertson was impressed, saying:

“She had somehow been able to take and synthesis these pieces of science and engineering and technology that I had never thought of.”

Robertson urged Elizabeth to follow her dream.  So she did: she launched a startup, tapping family connections to raise money.  By the end of 2006, she had raised almost $6 million. Her first employee was Shaunak Roy, who had a Ph.D. in chemical engineering.  Elizabeth had worked with Shaunak in Robertson’s lab.

Elizabeth and Shaunak dropped the patch idea and came up with a cartridge-and-reader system:

The patient would prick her finger to draw a small sample of blood and place it in a cartridge that looked like a thick credit card.  The cartridge would slot into a bigger machine called a reader.  Pumps inside the reader would push the blood through tiny channels in the cartridge and into little wells coated with proteins known as antibodies.  On its way to the wells, a filter would separate the blood’s solid elements, its red and white blood cells, from the plasma and let only the plasma through.  When the plasma came into contact with the antibodies, a chemical reaction would produce a signal that would be “read” by the reader and translated into a result.

Elizabeth envisioned placing the cartridges and readers in patient’s homes so that they could test their blood regularly.  A cellular antenna on the reader would send the test results to the computer of a patient’s doctor by way of a central server. This would allow the doctor to make adjustments to the patient’s medication quickly, rather than waiting for the patient to go get his blood tested at a blood-draw center or during his next office visit.

By late 2005, eighteen months after he’d come on board, Shaunak was beginning to feel like they were making progress.  The company had a prototype, dubbed the Theranos 1.0, and had grown to two dozen employees.  It also had a business model it hoped would quickly generate revenues: it planned to license its blood-testing technology to pharmaceutical companies to help them catch adverse drug reactions during clinical trials.

 

THE GLUEBOT

Edmund Ku was a talented engineer with a reputation for fixing tough problems.  When Ku interviewed with Elizabeth Holmes in early 2006, he felt inspired by her vision.

Elizabeth cited the fact that an estimated one hundred thousand Americans died each year from adverse drug reactions.  Theranos would eliminate all those deaths, she said.  It would quite literally save lives.

Ed’s job would be to turn the Theranos 1.0 prototype into a product that could be commercialized.  It soon became clear to Ed that this would be the most difficult engineering challenge he had faced.  The main challenge was that Elizabeth required that they use only a drop of blood pricked from the end of a finger.

Her obsession with miniaturization extended to the cartridge.  She wanted it to fit in the palm of a hand, further complicating Ed’s task.  He and his team spent months reengineering it, but they never reached a point where they could reliably reproduce the same test results from the same blood samples.

The quantity of blood they were allowed to work with was so small that it had to bediluted with a saline solution to create more volume.  That made what would otherwise have been relatively routine chemistry work a lot more challenging.

Adding another level of complexity, blood and saline weren’t the only fluids that had to flow through the cartridge.  The reactions that occurred when the blood reached the little wells required chemicals known as reagents.  Those were stored in separate chambers. 

All these fluids needed to flow through the cartridge in a meticulously choreographed sequence, so the cartridge contained little valves that opened and shut at precise intervals.  Ed and his engineers tinkered with the design and the timing of the valves and the speed at which the various fluids were pumped through the cartridge.

Another problem was preventing all those fluids from leaking and contaminating one another.

Meanwhile, having burned through its first $6 million, Theranos raised another $9 million.  A separate group of biochemists worked on the chemistry work.  But Elizabeth kept the two groups from communicating with one another.  She preferred to be the only one who could see the whole system.  This was far from ideal.  Ed couldn’t tell if problems they were trying to solve were caused by microfluidics, which was their focus, or chemistry, which the other group was responsible for.

One evening, Elizabeth told Ed that progress wasn’t fast enough.  She wanted the engineering department to run twenty-four hours a day.  Ed disagreed.

Ed noticed a quote cut out from an article sitting on Elizabeth’s desk.  It was from Channing Robertson:

“You start to realize you are looking in the eyes of another Bill Gates, or Steve Jobs.”

Carreyrou:

That was a high bar to set for herself, Ed thought.  Then again, if there was anyone who could clear it, it might just be this young woman.  Ed had never encountered anyone as driven and relentless.  She slept four hours a night and popped chocolate-coated coffee beans throughout the day to inject herself with caffeine.  He tried to tell her to get more sleep and to live a healthier lifestyle, but she brushed him off.

Around that time, Elizabeth was meeting regularly with the venture capitalist Don Lucas and also (less regularly) with Larry Ellison.  Lucas and Ellison had both invested in the recent Series B round that raised $9 million.  Carreyrou comments:

Ellison might be one of the richest people in the world, with a net worth of some $25 billion, but he wasn’t necessarily the ideal role model.  In Oracle’s early years, he had famously exaggerated his database software’s capabilities and shipped versions of it crawling with bugs.  That’s not something you could do with a medical device.

Since Ed had refused to make his engineering group work 24/7, Elizabeth had cooled towards him.  Soon she had hired a rival engineering group.  Ed’s group was put in competition with this new group.

Elizabeth had persuaded Pfizer to try the Theranos system in a pilot project in Tennessee.  The Theranos 1.0 devices would be put in people’s homes and they would test their blood every day.  Results would be sent wirelessly to the Theranos’s office in California, where they would be analyzed and then sent to Pfizer.  The bugs would have to be fixed before then.  Ed accompanied Elizabeth to Tennessee to start training doctors and patients on how to use the system.

When they got to Tennessee, the cartridges and the readers they’d brought weren’t functioning properly, so Ed had to spend the night disassembling and reassembling them on his bed in his hotel room.  He managed to get them working well enough by morning that they were able to draw blood samples from two patients and a half dozen doctors and nurses at a local oncology clinic.

The patients looked very sick.  Ed learned that they were dying of cancer.  They were taking drugs designed to slow the growth of their tumors, which might buy them a few more months to live.

Elizabeth declared the trip a success, but Ed thought it was too soon to use Theranos 1.0 in a patient study—especially on terminal cancer patients.

***

In August 2007, Elizabeth had everyone in the company gather.  Standing next to Elizabeth was Michael Esquivel, a lawyer.  Equivel announced that the company was suing three former employees—Michael O’Connell, Chris Todd, and John Howard—for stealing intellectual property.  Current employees were not to have any contact with these individuals.  And all documents and emails had to be preserved.  Moreover, the FBI had been called for assistance.

O’Connell held a postdoctorate in nanotechnology from Stanford.  He thought he had solved the microfluidic problems of the Theranos system.  O’Connell convinced Todd to form a company with him—Avidnostics.  O’Connell also spoke with Howard, who helped but decided not to join the company.

Elizabeth had always been very concerned about proprietary information getting out.  She required employees—as well as any who entered the Theranos office or did business with it—to sign nondisclosure agreements.

Meanwhile, the engineering teams vied to be first to solve the problems with Theranos 1.0.  Tony Nugent, an Irishman, was the head of the team competing with Ed’s team.

Tony decided that part of the Theranos value proposition should be to automate all the steps that bench chemists followed when they tested blood in a laboratory.  In order to automate, Tony needed a robot.  But he didn’t want to waste time building one from scratch, so he ordered a three-thousand-dollar glue-dispensing robot from a company in New Jersey called Fisnar.  It became the heart of the new Theranos system.

Soon Tony and team had built a smaller version of the robot that fit inside an aluminum box slightly smaller than a desktop computer tower.  Eventually they got the robot to follow the same steps a human chemist would.

First, it grabbed one of the two pipette tips and used it to aspirate the blood and mix it with diluents contained in the cartridge’s other tubes.  Then it grabbed the other pipette tip and aspirated the diluted blood with it.  This second tip was coated with antibodies, which attached themselves to the molecule of interest, creating a microscopic sandwich.

Therobot’s last step was to aspirate reagents from yet another tube in the cartridge.  When the reagents came into contact with the microscopic sandwiches, a chemical reaction occurred that emitted a light signal.  An instrument inside the reader called a photomultiplier tube then translated the light signal into an electric current.

The molecule’s concentration in the blood—what the test sought to measure—could be inferred from the power of the electrical current, which was proportional to the intensity of the light.

This blood-testing technique was known as chemiluminescent immunoassay.  (In laboratory speak, the word “assay” is synonymous with “blood test.”)  The technique was not new: it had been pioneered in the early 1980s by a professor at Cardiff University.  But Tony had automated it inside a machine that, though bigger than the toaster-size Theranos 1.0, was still small enough to make Elizabeth’s vision of placing it in patients’ homes possible.  And it only required about 50 microliters of blood.  That was more than 10 microliters Elizabeth initially insisted upon, but it still amounted to just a drop.

Once they had a functioning prototype—it worked much better than the system Ed was working on—Elizabeth suggested they call it the Edison (since everything else had failed).  Elizabeth decided to use the Edison instead of the microfluidic system.  Carreyrou points out the irony of this decision, given that the company had just filed a lawsuit to protect the intellectual property associated with the microfluidic system.  Soon thereafter, Ed and his team were let go.

Carreyrou continues:

Shaunak followed Ed out the door two weeks later, albeit on friendlier terms.  The Edison was at its core a converted glue robot and that was a pretty big step down from the lofty vision Elizebeth had originally sold him on.  He was also unsettled by the constant staff turnover and the lawsuit hysteria.

Although Elizabeth was excited about the Edison, it was a long way from a finished product.

 

APPLE ENVY

Elizabeth worshipped Steve Jobs and Apple.  In the summer of 2007, she started recruiting employees of Apple.  Ana Arriola, a product designer who worked on the iPhone, was one such recruit.

…Elizabeth told Ana she envisioned building a disease map of each person through Theranos’s blood tests.  The company would then be able to reverse engineer illnesses like cancer with mathematical models that would crunch the blood data and predict the evolution of tumors.

Ana and (later) her wife Corrine were impressed enough that Ana decided to leave behind fifteen thousand Apple shares.  She was hired as Theranos’s chief design officer.  Elizabeth wanted a software touchscreen like the iPhone’s for the Edison.  And she wanted a sleek outer case.

Since fans like Channing Robertson and Don Lucas were starting to compare Elizabeth to Steve Jobs, Ana thought she should look the part.  Soon Elizabeth was wearing a black turtleneck and black slacks most of the time.

Elizebeth’s idealism seemed like a good thing.  But there continued to be other aspects of working at Theranos that seemed stifling (to say the least).  Different groups were prevented from sharing information and working together.  Moreover, employees knew they were being spied on—including what they did on their computer and what they did on Facebook. 

Also, one of Elizabeth’s assistants kept track of how many hours each employee worked.  Elizabeth had dinner catered—it arrived at 8 or 8:30 each night—in order to encourage people to put in more hours.

Finally, people were constantly getting fired from Theranos.

One person Elizabeth recruited to Theranos was Avie Tevanian.  Avie was one of Steve Job’s closest friends.  Avie had worked with Jobs and NeXT, and then went with Jobs to Apple in 1997.  Avie was the head of software engineering.  After an arduous decade, Avie retired.  Elizabeth convinced Avie to join the Theranos board.  Avie invested $1.5 million into the company in the 2006 offering.

The first couple of board meetings Avie attended had been relatively uneventful, but, by the third one, he’d begun to notice a pattern.  Elizabeth would present increasingly rosy projections based on the deals she said Theranos was negotiating with pharmaceutical companies, but the revenues wouldn’t materialize.  It didn’t help that Henry Mosley, the chief financial officer, had been fired soon after Avie became a director.  At the last board meeting he’d attended, Avie had asked more pointed questions about the pharmaceutical deals and been told they were held up in legal review.  When he’d asked to see the contracts, Elizabeth had said she didn’t have any copies readily available.

There were also repeated delays with the product’s rollout and the explanation for what needed to be fixed kept changing.  Avie didn’t pretend to understand the science of blood-testing; his expertise was software.  But if the Theranos system was in the final stages of fine-tuning as he’d been told, how could a completely different technical issue be the new holdup every quarter?  That didn’t sound to him like a product that was on the cusp of commercialization.

Avie started asking more questions at the board meetings.  Soon thereafter, Don Lucas contacted Avie and informed him that Elizabeth was upset with his behavior and wanted him to leave the board.  Avie was surprised because he was just doing his duty as a board member.  Avie decided to look at all the material he’d been given over the previous year, including investment material. 

As he read them over, he realized that everything about the company had changed in the space of a year, including Elizabeth’s entire executive team.  Don needed to see these, he thought.

Ana Arriola was also growing concerned.  One morning, Ana brought up a question to Elizabeth.  Given that the technology wasn’t working, why not pause the Tennessee study in order to fix the bugs first?  They could always restart the study later once the product was functional.  Elizabeth replied that Pfizer and every other big pharma company wanted her blood-testing system and Theranos was going to be a great company.  Then Elizabeth suggested to Ana that she might be happier elsewhere. 

Ana knew it wasn’t right to use an unreliable blood-test system in the Tennessee study.  Later that same day, she resigned.

When Avie showed the material he’d gathered to Don, Don suggested to Avie that he resign.  Avie was surprised that Don didn’t seem interested in the material.  But Avie realized that he’d retired from Apple for good reason (to spend more time with his family).  He didn’t need extra aggravation.  So he told Don he would resign.

Don then brought up one more thing to Avie.  Shaunak Roy was leaving and was selling his shares to Elizabeth.  She needed the board to waive the company’s rights to repurchase Shaunak’s stock.  Avie didn’t think that was a good idea.  Don then said he wanted Avie to waive his own right as a board member to purchase the stock.

Avie was starting to get upset.  He told Don to have Theranos’s general, Michael Esquivel, to send him the information.  After reading it, Avie thought it was clear that he could buy some of Shaunak’s stock.  Avie decided to do so and informed Esquivel.  That prompted an argument.

At 11:17 p.m. on Christmas Eve, Esquivel sent Avie an email accusing him of acting in “bad faith” and warned him that Theranos was giving serious consideration to suing him for breach of his fiduciary duties as a board member and for public disparagement of the company.

Avie was astonished.  Not only had he done no such things, in all his years in Silicon Valley he had never come close to being threatened with a lawsuit.  All over the Valley, he was known as a nice guy.  A teddy bear.  He didn’t have any enemies.  What was going on here? 

Avie spoke with a friend who was a lawyer, who asked Avie if, given what he now knew, he really wanted to buy more shares in the company.  No, thought Avie.  But he did write a parting letter to Don summarizing his concerns.  The letter concluded:

“I do hope you will fully inform the rest of the Board as to what happened here.  They deserve to know that by not going along 100% ‘with the program’ they risk retribution from the Company/Elizabeth.”

 

GOODBYE EAST PALY

In early 2008, Theranos moved to a new location on Hillview Avenue in Palo Alto.  It was a drastic improvement over their previous location in East Palo Alto.  That area—east of Highway 101 (Bayshore Freeway)—was much poorer and had once been the country’s murder capital.

Matt Bissel, head of IT, was in charge of the move.  At 4 o’clock in the afternoon the day before the movers were scheduled to come, Matt was pulled into a conference room.  Elizabeth was on the line from Switzerland.  She told Matt she’d just learned that Theranos would be charged an extra month’s rent if they waited until tomorrow.  She told Matt to call the moving company immediately to see if they could do it.  No way.  But Elizabeth kept pushing.  Someone pointed out that the blood samples wouldn’t be kept at the right temperature if they were moved immediately.  Elizabeth said they could put them in refrigerated trucks.

Finally, Matt got Elizabeth to slow down after pointing out that they would still have to do an inspection with state officials to prove that they had properly disposed of any hazardous materials.  That meant no new tenant could move in until then.

Matt greatly admired Elizabeth as one of the smartest people he’d ever met, and as an energizing leader.  But he was starting to grow worried about some aspects of the company.

One aspect of Matt’s job had become increasingly distasteful to him.  Elizabeth demanded absolute loyalty from her employees and if she sensed that she no longer had it from someone, she could turn on them in a flash.  In Matt’s two and a half years at Theranos, he had seen her fire some thirty people, not counting the twenty or so employees who lost their jobs at the same time as Ed Ku when the microfluidic platform was abandoned.

Every time Elizabeth fired someone, Matt had to assist with terminating the employee…In some instances, she asked him to build a dossier on the person that she could use for leverage.

Matt was bothered in particular about how John Howard was treated.

When Matt reviewed all the evidence assembled for the Michael O’Connell lawsuit, he didn’t see anything proving that Howard had done anything wrong.  He’d had contact with O’Connell but he’d declined to join his company.  Yet Elizabeth insisted on connecting the dots in a certain way and suing him too, even though Howard been one of the first people to help her when she dropped out of Stanford, letting her use the basement of his house in Saratoga for experiments in the company’s early days.

Matt decided it was a good time to launch his own IT company.  When he informed Elizabeth, she couldn’t believe it.  She offered him a raise and a promotion, but he turned her down.  Then she grew very cold.  She offered one of Matt’s colleagues, Ed Ruiz, Matt’s position if he would look through Matt’s files and emails.  Ed was good friends with Matt and refused. (There was nothing to find anyway.)  A few months after Matt left, Ed decided to work for Matt’s new company.

Meanwhile, Aaron Moore and Mike Bauerly wanted to test two ofthe Edison prototypes built by Tony Nugent and Dave Nelson.  This was informal “human factors” research to see how people reacted.

Aaron took photos with his digital camera to document what they were doing.  The Eve Behar cases weren’t ready yet, so the devices had a primitive look.  Their temporary cases were made from gray aluminum plates bolted together.  The front plate tilted upward like a cat door to let the cartridge in.  A rudimentary software interface sat atop the cat door at an angle.  Inside, the robotic arm made loud, grinding sounds.  Sometimes, it would crash against the cartridge and the pipette tips would snap off.  The overall impression was that of an eighth-grade science project.

Aaron and Mike visited their friends’ offices to do the tests.

As the day progressed, it became apparent that one pinprick often wasn’t enough to get the job done.  Transferring the blood to the cartridge wasn’t the easiest of procedures.  The person had to swab his finger with alcohol, prick it with the lancet, apply the transfer pen to the blood that bubbled up from the finger to aspirate it, and then press on the transfer pen’s plunger to expel the blood into the cartridge.  Few people got it right on their first try.  Aaron and Mike kept having to ask their test subjects to prick themselves multiple times.  It got messy. There was blood everywhere.

This confirmed Aaron was worried about: A fifty-five-year-old patient in his or her home was going to have trouble.  Aaron passed on his concerns to Tony and Elizabeth, but they didn’t think it was important.  Aaron was getting disillusioned.

A bit later, Todd Surdey was hired to run sales and marketing.  One of Todd’s two subordinates was on the East Coast: Susan DiGiaimo.  Susan had accompanied Elizabeth on quite a few sales pitches to drug makers.  Susan had been uncomfortable about Elizabeth’s very lofty promises.

Todd asked Susan about Elizabeth’s revenue projections.  Susan replied that they were vastly overinflated.

Moreover, no significant revenues would materialize unless Theranos proved to each partner that its blood system worked.  To that effect, each deal provided for an initial tryout, a so-called validation phase…

The 2007 study in Tennessee was the validation phase of the Pfizer contract.  Its objective was to prove that Theranos could help Pfizer gauge cancer patients’ response to drugs by measuring the blood concentration of three proteins the body produces in excess when tumors grow.  If Theranos failed to establish any correlation between the patients’ protein levels and the drugs, Pfizer could end their partnership and any revenue forecast Elizabeth had extrapolated from the deal would turn out to be fiction.

Susan also shared with Todd that she had never seen any validation data.  And when she went on demonstrations with Elizabeth, the devices often malfunctioned.  A case in point was the one they’d just conducted at Novartis.  After the first Novartis demo in late 2006 during which Tim Kemp had beamed a fabricated result from California to Switzerland, Elizabeth had continued to court the drug maker and had arranged a second visit to its headquarters in January 2008.

The night before that second meeting, Susan and Elizabeth had pricked their fingers for two hours in a hotel in Zurich to try to establish some consistency in the test results they were getting, to no avail. When they showed up at Novartis’s Basel offices the next morning, it got worse: all three Edison readers produced error messages in front of a room full of Swiss executives.  Susan was mortified, but Elizabeth kept her composure and blamed a minor technical glitch.

Based on the intel he was getting from Susan and from other employees in Palo Alto, Todd became convinced that Theranos’s board was being misled about the company’s finances and the state of its technology.

Todd brought his concerns to Michael Esquivel, the company’s general counsel.  Michael had been harboring his own suspicions.  In March 2008, Todd and Michael approached Tom Brodeen, a Theranos board member.  Since he was relatively new, he said they should raise their concerns with Don Lucas, the board’s chairman.  So they did.

This time, Don Lucas had to take the matter seriously.

Lucas convened an emergency meeting of the board in his office on Sand Hill Road.  Elizabeth was asked to wait outside the door while the other directors—Lucas, Brodeen, Channing Robertson, and Peter Thomas, the founder of an early stage venture capital firm called ATAventures—conferred inside.

After some discussion, the four men reached a consensus: they would remove Elizabeth as CEO.  She had proven herself too young and inexperienced for the job.  Tom Brodeen would step in to lead the company for a temporary period until a more permanent replacement could be found.  They called in Elizabeth to confront her with what they had learned and inform her of their decision.

But then something extraordinary happened.

Over the course of the next two hours, Elizabeth convinced them to change their minds.  She told them she recognized there were issues with her management and promised to change.  She would be more transparent and responsive going forward.  It wouldn’t happen again.

A few weeks later, Elizabeth fired Todd and Michael.  Soon thereafter, Justin Maxwell, a friend of Aaron and Mike’s, decided to resign.  His resignation email included this:

believe in the people who disagree with you… Lying is a disgusting habit, and it flows through the conversations here like its our own currency.  The cultural disease here is what we should be curing… I mean no ill will towards you, since you believe in what I was doing and hoped I would succeed at Theranos.

A few months later, Aaron Moore and Mike Bauerly resigned.

 

THE CHILDHOOD NEIGHBOR 

Richard Fuisz was a medical inventor and entrepreneur.  He was following what Elizabeth was doing at Theranos.  The Fuisz and Holmes families had been friends for two decades.

Elizabeth’s mother, Noel, and Richard’s wife, Lorraine, had developed a close friendship.  But the husbands weren’t as close.  This may have been because Chris Holmes was on a government salary, while Richard Fuisz was a successful businessman who liked to flaunt it.

Money was indeed a sore point in the Holmes household.  Chris’s grandfather, Christian Holmes II, had depleted his share of the Fleischmann fortune by living a lavish and hedonistic lifestyle on an island in Hawaii, and Chris’s father, Christian III, had frittered away what was left during an unsuccessful career in the oil business.

Carreyrou later explains:

Richard Fuisz was a vain and prideful man.  The thought that the daughter of longtime friends and former neighbors would launch a company in his area of expertise and that they wouldn’t ask for his help or even consult him deeply offended him. 

Carreyrou again:

Fuisz had a history of taking slights personally and bearing grudges.  The lengths he was willing to go to get even with people he perceived to have crossed him is best illustrated by his long and protracted feud with Vernon Loucks, the CEO of hospital supplies maker Baxter International.

Fuisz traveled frequently to the Middle East in the 1970s and early 1980s because it was the biggest market for his medical film business, Medcom.  On one of these trips, he ran into Loucks.  Over dinner, Loucks offered to buy Medcom for $53 million.  Fuisz agreed. 

Fuisz was supposed to be head of the new Baxter subsidiary for three years, but Loucks dismissed him after the acquisition closed.  Fuisz sued Baxter for wrongful termination, asserting that Loucks fired him for refusing to pay a $2.2 million bribe to a Saudi firm to remove Baxter from an Arab blacklist of companies doing business with Israel.  Carreyrou:

The two sides reached a settlement in 1986, under which Baxter agreed to pay Fuisz $800,000.  That wasn’t the end of it, however.  When Fuisz flew to Baxter’s Deerfield, Illinois, headquarters to sign the settlement, Loucks refused to shake his hand, angering Fuisz and putting him back on the warpath.

In 1989, Baxter was taken off the Arab boycott list, giving Fuisz an opening to seek his revenge.  He was leading a double life as an undercover CIA agent by then, having volunteered his services to the agency a few years earlier after coming across one of its ads in the classified pages of the Washington Post.

Fuisz’s work for the CIA involved setting up dummy corporations throughout the Middle East that employed agency assets, giving them a non-embassy cover to operate outside the scrutiny of local intelligence services.  One of the companies supplied oil-rig operators to the national oil company of Syria, where he was particularly well-connected.

Fuisz suspected Baxter had gotten itself back in Arab countries’ good graces through chicanery and set out to prove it using his Syrian connections. He sent a female operative he’d recruited to obtain a memorandum kept on file in the offices of the Arab League committee in Damascus that was in charge of enforcing the boycott.  It showed that Baxter had provided the committee detailed documentation about its recent sale of an Israeli plant and promised it wouldn’t make new investments in Israel or sell the country new technologies.  This put Baxter in violation of a U.S. anti-boycott law, enacted in 1977, that forbade American companies from participating in any foreign boycott or supplying blacklist officials any information that demonstrated cooperation with the boycott.

Fuisz sent a copy of the memo to Baxter’s board of directors and another copy to the Wall Street Journal.  The Journal published a front-page story.  Fuisz then was able to get copies of letters Baxter’s general counsel had written to a general in the Syrian army.  These letters confirmed the memo.

The revelations led the Justice Department to open an investigation.  In March 1993, Baxter was forced to plead guilty to a felony charge of violating the anti-boycott law and to pay $6.6 million in civil and criminal fines.  The company was suspended from new federal contracts for four months and barred from doing business in Syria and Saudi Arabia for two years.  The reputational damage also cost it a $50 million contract with a big hospital group.

Fuisz was upset, however, that Loucks still remained CEO.  So he came up with another attack.  Loucks was a Yale alumnus and served as trustee of Yale Corporation, the university’s governing body.  He also chaired the university’s fund-raising campaign.  Commencement ceremonies were coming up that May.

Through his son Joe, who had graduated from Yale the year before, Fuisz got in touch with a student named Ben Gordon, who was the president of Yale Friends of Israel association.  Together, they organized a graduation day protest featuring “Loucks Is Bad for Yale” signs and leaflets.  The crowning flourish was a turboprop plane Fuisz hired to fly over the campus trailing a banner that read, “Resign Loucks.”

Three months later, Loucks stepped down as Yale trustee.

All of that said, Fuisz’s initial interest in Theranos’s technology came more from opportunism than any desire for revenge.  Fuisz had made quite a bit of money by patenting inventions he thought other companies would want at some point.  Carreyrou:

One of his most lucrative plays involved repurposing a cotton candy spinner to turn drugs into fast-dissolving capsules.  The idea came to him when he took his daughter to a country fair in Pennsylvania in the early 1990s.  He later sold the public corporation he formed to house the technology to a Canadian pharmaceutical company for $154 million and personally pocketed $30 million from the deal.

Fuisz listened to an interview Elizabeth did for NPR’s “Biotech Nation,” in May 2005.  In that interview, Elizabeth explained how her blood-test system could be used for at-home monitoring of adverse reactions to drugs.

…But as a trained physician, he also spotted a potential weakness he could exploit.  If patients were going to test their blood at home with the Theranos device to monitor how they were tolerating the drugs they were taking, there needed to be a built-in mechanism that would alert their doctors when the results came back abnormal.

He saw a chance to patent that missing element, figuring there was money to be made down the road, whether from Theranos or someone else.  His thirty-five years of experience patenting medical inventions told him such a patent might eventually command up to $4 million for an exclusive license.

Fuisz filed a fourteen-page patent application with he U.S. Patent and Trademark office on April 24, 2006. It didn’t claim to invent new technology, but to combine existing technologies—wireless data transmission, computer chips, and bar codes—into a physician alert mechanism that could be part of at-home blood-testing devices.  The application said clearly that it was  targeting the company Theranos.

Meanwhile, for other reasons, the Fuisz and Holmes families grew apart.  By the time Elizabeth became aware of Richard Fuisz’s patent, the two families were no longer on speaking terms.

 

SUNNY     

Chelsea Burkett and Elizabeth had been friends at Stanford.  Elizabeth recruited Chelsea to Theranos.  Chelsea found Elizabeth to be very persuasive:

She had this intense way of looking at you while she spoke that made you believe in her and want to follow her.

About the same time Chelsea joined Theranos, Ramesh “Sunny” Balwani came on board as a senior Theranos executive.  All that Chelsea knew was that Sunny was Elizabeth’s boyfriend and that they were living together in a Palo Alto apartment.  Sunny immediately asserted himself and seemed omnipresent.

Sunny was a force of nature, and not in a good way.  Though only about five foot five and portly, he made up for his diminutive stature with an aggressive, in-your-face management style.  His thick eyebrows and almost-shaped eyes, set above a mouth that drooped at the edges and a square chin, projected an air of menace.  He was haughty and demeaning towards employees, barking orders and dressing people down.

Chelsea took an immediate dislike to him even though he made an effort to be nicer to her in deference to her friendship with Elizabeth.  She didn’t understand what her friend saw in this man, who was nearly two decades older than she was and lacking in the most basic grace and manners.  All her instincts told her Sunny was bad news, but Elizabeth seemed to have the utmost confidence in him.

Elizabeth and Sunny had met in Beijing when Elizabeth was in her third year of a Stanford program that taught students Mandarin.  Apparently, Elizabeth had been bullied by some of the students and Sunny came to her defense.

Sunny was born and raised in Pakistan.  He pursued his undergraduate studies in the U.S.  Then he worked for a decade as a software engineer for Lotus and Microsoft.  In 1999, Sunny joined CommerceBid.com, which was developing software that would have suppliers bid against one another in an auction.  The goal was to achieve economies of scale and lower prices.

In November 1999, a few months after Sunny joined CommerceBid as president and chief technology officer, the company was acquired for $232 million in cash and stock.  Carreyrou:

It was a breathtaking price for a company that had just three clients testing its software and barely any revenues.  As the company’s second-highest-ranking executive, Sunny pocketed more than $40 million.  His timing was perfect.  Five months later, the dot-com bubble popped and the stock market came crashing down. Commerce One [the company that acquired CommerceBid] eventually filed for bankruptcy.

Yet Sunny didn’t see himself as lucky.  In his mind, he was a gifted businessman and the Commerce One windfall was a validation of his talent.  When Elizabeth met him a few years later, she had no reason to question that.  She was an impressionable eighteen-year-old girl who saw in Sunny what she wanted to become: a successful and wealthy entrepreneur.  He became her mentor, the person who would teach her about business in Silicon Valley.

In 2004, the IRS forced Sunny to pay millions in back taxes after he had tried to use a tax shelter. Sunny liked to flaunt his wealth.  He drove a black Lamborghini Gallardo and a black Porsche 911.  Carreyrou writes:

He wore white designer shirts with puffy sleaves, acid-washed jeans, and blue Gucci loafers.  His shirts’ top three buttons were always undone, causing his chest hair to spill out and revealing a thin gold chain around his neck.  A pungent scent of cologne emanated from him at all times.  Combined with the flashy cars, the overall impression was of someone heading out to a nightclub rather than to the office.

Sunny was supposed to be an expert in software.  He even bragged that he’d written a million lines of code.  Some employees thought that was an absurd claim.  (Microsoft software engineers had written the Windows operating system at a rate of one thousand lines of code per year, notes Carreyrou.)

Carreyrou adds:

There was also the murky question of what she told the board about their relationship.  When Elizabeth informed Tony that Sunny was joining the company, Tony asked her point-blank whether they were still a couple.  She responded that the relationship was over.  Going forward, it was strictly business, she said.  But that would prove not to be true.

Carreyrou continues:

When Elizabeth pitched pharmaceutical executives now, she told them that Theranos would forecast how patients would react to the drugs they were taking.  Patients’ test results would be input into a proprietary computer program the company had developed.  As more results got fed into the program, its ability to predict how markers in the blood were likely to change during treatment would become better and better, she said.

It sounded cutting-edge, but there was a catch: the blood-test results had to be reliable for the computer program’s predictions to have any value… Theranos was supposed to help Centocor [in Antwerp, Belgium] to assess how patients were responding to an asthma drug by measuring a biomarker in their blood called allergen-specific immunoglobulin E, or IgE, but the Theranos devices seemed very buggy to Chelsea.  There were frequent mechanical failures.  The cartridges either wouldn’t slot into the readers properly or something inside the readers would malfunction.  Even when the devices didn’t break down, it could be a challenge coaxing any kind of output from them.

Sunny always blamed the wireless connection.  That was true sometimes, but there were other things that could interfere.  Nearly all blood tests require some dilution, but too much dilution made it harder for Theranos to get accurate results.

The amount of dilution the Theranos system required was greater than usual because of the small size of the blood samples Elizabeth insisted on.

Furthermore, to function properly, the Edisons required the ambient temperature to be exactly 34 degrees Celsius.  Two 11-volt heaters built into the reader tried to maintain that temperature during a blood test.  But in colder settings, including some hospitals in Europe, the temperature couldn’t be maintained.

Meanwhile, Pfizer had ended its collaboration with Theranos because it was not impressed by the results of the Tennessee validation study.

The study had failed to show any clear link between drops in the patients’ protein levels and the administration of the antitumor drugs.  And the report had copped to some of the same snafus Chelsea was now witnessing in Belgium, such as mechanical failures and wireless transmission errors.

When Chelsea returned from her three-week trip to Europe, she found that Elizabeth and Sunny were now focused on Mexico, where a swine flu epidemic had been raging.  Seth Michelson, Theranos’ chief scientific officer, had suggested an idea to Elizabeth.

Seth had told Elizabeth about a math model called SEIR (Susceptible, Exposed, Infected, and Resolved) that he thought could be adapted to predict where the swine flu virus would spread next.  For it to work, Theranos would need to test recently infected patients and input their blood-test results into the model.  That meant getting the Edison readers and cartridges to Mexico. Elizabeth envisioned putting them in the beds of pickup trucks and driving them to the Mexican villages on the front lines of the outbreak.

Because Chelsea was fluent in Spanish, she and Sunny were sent.  Elizabeth used her family connections in order to get authorization to use the experimental medical device in Mexico.

Once again, things did not go smoothly.  Frequently, the readers flashed error messages, or the result that came back from Palo Alto was negative for the virus when it should have been positive.  Some of the readers didn’t work at all.  And Sunny continued to blame the wireless transmission.

Chelsea grew frustrated and miserable.  She questioned what she was even doing there.  Gary Frenzel and some of the other Theranos scientists had told her that the best way to diagnose H1N1, as the swine flu virus was called, was with a nasal swab and that testing for it in blood was a questionable utility.  She’d raised this point with Elizabeth before leaving, but Elizabeth had brushed it off. “Don’t listen to them,” she’d said of the scientists.  “They’re always complaining.”

At the time, Theranos was struggling financially:

The $15 million Theranos had raised in its first two funding rounds was long gone and the company had already burned through the $32 million Henry Mosley had been instrumental in bringing in during its Series C round in late 2006.  The company was being kept afloat by a loan Sunny had personally guaranteed.

Meanwhile, Sunny was also traveling to Thailand to set up another swine flu testing outpost.  The epidemic had spread to Asia, and the country was one of the region’s hardest hit with tens of thousands of cases and more than two hundred deaths.  But unlike in Mexico, it wasn’t clear that Theranos’s activities in Thailand were sanctioned by local authorities.  Rumors were circulating among employees that Sunny’s connections there were shady and that he was paying bribes to obtain blood samples from infected patients. When a colleague of Chelsea’s in the client solutions group named Stefan Hristu quit immediately upon returning from a trip to Thailand with Sunny in January 2010, many took it to mean the rumors were true.

On the whole, Sunny had created a culture of fear with his bullying behavior.  The high rate of firings continued, and Sunny had taken charge of it.  Remaining employees started to say, “Sunny disappeared him” whenever Sunny fired someone.

The scientists, especially, were afraid of Sunny. One of the only ones who stood up to him was Seth Michelson.  A few days before Christmas, Seth had gone out and purchased polo shirts for his group. Their color matched the green of the company logo and they had the words “Theranos Biomath” emblazoned on them. Seth thought it was a nice team-building gesture and paid for it out of his own pocket.

When Sunny saw the polos, he got angry.  He didn’t like that he hadn’t been consulted and he argued that Seth’s gift to his team made the other managers look bad.  Earlier in his career, Seth had worked at Roche, the big Swiss drug maker, where he’d been in charge of seventy people and an annual budget of $25 million.  He decided he wasn’t going to let Sunny lecture him about management.  He pushed back and they got into a yelling match.

Soon thereafter, Seth found another job at Genomic Health in Redwood City.  When he went to give his resignation letter to Elizabeth, Sunny was there.  He read the letter and threw it in Seth’s face, shouting, “I won’t accept this!”

Seth shouted back: “I have news for you, sir: in 1863, President Lincoln freed the slaves!”

Sunny’s response was to throw him out of the building. It was weeks before Seth was able to retrieve his math books, scientific journals, and the pictures of his wife on his desk.  He had to enlist the company’s new lawyer, Jodi Sutton, and a security guard to help him pack his things late on a weeknight when Sunny wasn’t around.

Sunny also got into a yelling match with Tony Nugent.  Chelsea attempted to get through to Elizabeth about Sunny, but she was unable to.

Chelsea wanted to quit, but still wasn’t sure.  Then one day the Stanford student with the family connections to Mexico stopped by with his father, who was dealing with a cancer scare of some sort.   Elizabeth and Sunny persuaded him to allow Theranos to test his blood for cancer biomarkers.

Chelsea was appalled.  The validation study in Belgium and the experiments in Mexico and Thailand were one thing.  Those were supposed to be for research purposes only and to have no bearing on the way patients were treated.  But by encouraging someone to rely on a Theranos blood test to make an important medical decision was something else altogether.  Chelsea found it reckless and irresponsible.

She became further alarmed when not long afterward Sunny and Elizabeth began circulating copies of the requisition forms doctors used to order blood tests from laboratories and speaking excitedly about the great opportunities that lay in consumer testing.

I’m done, Chelsea thought to herself.  This has crossed too many lines.

[…]

Chelsea also worried about Elizabeth.  In her relentless drive to be a successful startup founder, she had built a bubble around herself that was cutting her off from reality.  And the only person she was letting inside was a terrible influence.  How could her friend not see that?

 

DR. J

Dr. J was Jay Rosen’s nickname.  Rosen is a doctor who is a member of Walgreens’s innovation team, whose goal is to find ideas and technologies that could create growth.

In January 2010, Theranos approached Walgreens with an email stating that it had developed small devices capable of running any blood test from a few drops pricked from a finger in real time and for less than half the cost of traditional laboratories.  Two months later, Elizabeth and Sunny traveled to Walgreens’s headquarters in the Chicago suburb of Deerfield, Illinois, and gave a presentation to a group of Walgreens executives.  Dr. J, who flew up from Pennsylvania for the meeting, instantly recognized the potential for the Theranos technology.  Bringing the startup’s machines inside Walgreens stores could open up a big new revenue stream for the retailer and be the game changer it had been looking for, he believed.

…The picture Elizabeth presented at the meeting of making blood tests less painful and more widely available so they could become an early warning system against disease deeply resonated with him.

On August 24, 2010, a Walgreens delegation arrived at the Theranos office in Palo Alto for a two-day meeting.  Kevin Hunter was the leader of a small lab consulting firm Colaborate.  Walgreens had hired him to evaluate and launch a partnership it was setting up with the startup.  Hunter’s father, grandfather, and great-grandfather had all been pharmacists.

Walgreens and Theranos had signed a preliminary contract.  Walgreens would prepurchase up to $50 million worth of Theranos cartridges and loan the startup $25 million.  If the pilot went well, the companies would expand their partnership nationwide.

Hunter asked to see the lab, but Elizabeth said later if there was time.  Hunter asked again.  Elizabeth pulled Dr. J aside.  Then Dr. J informed Hunter that they wouldn’t see the lab yet.

Theranos had told Walgreens it had a commercially ready laboratory and had provided it with a list of 192 different blood tests it said its proprietary devices could handle.  In reality, although there was a lab downstairs, it was just an R&D lab where Gary Frenzel and his team of biochemists conducted their research. Moreover, half of the tests on the list couldn’t be performed as chemiluminescent immunoassays, the testing technique the Edison system relied on.  These required different testing methods beyond the Edison’s scope.

Carreyrou writes:

Hunter was beginning to grow suspicious.  With her black turtleneck, her deep voice, and the green kale shakes she sipped on all day, Elizabeth was going to great lengths to emulate Steve Jobs, but she didn’t seem to have a solid understanding of what distinguished different types of blood tests.  Theranos had also failed to deliver on his two basic requests: to let him see its lab and to demonstrate a live vitamin D test on its device. Hunter’s plan had been to have Theranos test his and Dr. J’s blood, then get retested at Stanford Hospital that evening and compare results.  He’d even arranged for a pathologist to be on standby at the hospital to write the order and draw their blood.  But Elizabeth claimed she’d been given too little notice even though he’d made the request two weeks ago.

Despite Hunter’s suspicions, Dr. J and the Walgreens CFO, Wade Miquelon, continued to be big fans of Elizabeth.

In September 2010, Elizabeth and Sunny met with Walgreens executives at the company’s headquarters in Deerfield.  Elizabeth and Sunny suggested they do blood tests on the executives.  Hunter wasn’t at the meeting, but he heard about the blood tests later.  He thought it was a good opportunity to see how the technology performed.

Hunter asked about the blood-test results a few days later on the weekly video conference call the companies were using as their primary mode of communication.  Elizabeth responded that Theranos could only release the results to a doctor.  Dr J…reminded everyone that he was a trained physician, so why didn’t Theranos go ahead and send him the results?  They agreed that Sunny would follow up separately with him.

A month passed and still no results.

There was another issue, too. Theranos had suddenly changed its regulatory strategy.  Initially Theranos said the blood tests would qualify as “waived” under the Clinical Laboratory Improvement Amendments, a 1988 federal law that covered laboratories.

CLIA-waved tests usually involved simple laboratory procedures that the Food and Drug Administration had cleared for home use.

Now, Theranos was changing its tune and saying the tests it would be offering in Walgreens stores were “laboratory-developed tests.”  It was a big difference: laboratory-developed tests lay in a gray zone between the FDA and another federal health regulator, the Centers for Medicare and Medicaid Services.  CMS, as the latter agency was known, exercised oversight of clinical laboratories under CLIA, while the FDA regulated the diagnostic equipment that laboratories bought and used for their testing. But no one closely regulated tests that labs fashioned with their own methods.  Elizabeth and Sunny had a testy exchange with Hunter over the significance of the change.  They maintained that all the big laboratory companies mostly used laboratory-developed tests, which Hunter knew not to be true.

Hunter argued that it was now even more important to make sure Theranos’s tests were accurate.  He suggested a fifty-patient study, which he could easily arrange.  Hunter noticed that Elizabeth became defensive immediately.  She said they didn’t want to do it “at this time,” and she quickly changed the subject.

After they hung up, Hunter took aside Renaat Van den Hooff, who was in charge of the pilot on the Walgreens side, and told him something just wasn’t right.  The red flags were piling up.  First, Elizabeth had denied him access to their lab.  Then she’d rejected his proposal to embed someone with them in Palo Alto.  And now she was refusing to do a simple comparison study.  To top it all off, Theranos had drawn the blood of the president of Walgreen’s pharmacy business, one of the company’s most senior executives, and failed to give him a test result!

Van den Hooff told Hunter:

“We can’t not pursue this.  We can’t risk a scenario where CVS has a deal with them in six months and it ends up being real.”

Almost everything Walgreens did was done with its rival CVS in mind.

Theranos had cleverly played on this insecurity.  As a result, Walgreens suffered from a severe case of FoMO—the fear of missing out.

There were two more things Theranos claimed were proof that its technology works.  First, there was clinical trial work Theranos had done with pharmaceutical companies.  Hunter had called the pharmaceutical companies, but hadn’t been able to reach anyone who could verify Theranos’s claims.  Second, Dr. J had commissioned Johns Hopkins University’s medical school to do a review of Theranos’s technology.

Hunter asked to see the Johns Hopkins review.  It was a two-page document.

When Hunter was done reading it, he almost laughed. It was a letter dated April 27, 2010, summarizing a meeting Elizabeth and Sunny had had with Dr. J and five university representatives on the Hopkins campus in Baltimore.  It stated that they had shown the Hopkins team “proprietary data on test performance” and that Hopkins had deemed the technology “novel and sound.”  But it also made clear that the university had conducted no independent verification of its own.  In fact, the letter included a disclaimer at the bottom of the second page: “The materials provided in no way signify an endorsement by Johns Hopkins Medicine to any product or service.”

In addition to Walgreens, Theranos also tried to get Safeway as a partner.  Elizabeth convinced Safeway’s CEO, Steve Burd, to do a deal.  Safeway loaned Theranos $30 million.  Safeway also committed to a massive renovation project of its stores, creating new clinics where customers could have their blood tested on Theranos devices.  Burd saw Elizabeth as a genius.

In early 2011, Hunter was informed that Elizabeth and Sunny no longer wanted him on the calls or in meetings between Theranos and Walgreens.  Hunter asked: Why Walgreens was paying him $25,000 a month to look out for its interests if he couldn’t do his job, which includes asking tough questions? 

 

THE MINILAB  

Elizabeth had told Walgreens and Safeway that Theranos’s technology could perform hundreds of tests on small blood samples. 

The truth was that the Edison system could only do immunoassays, a type of test that uses antibodies to measure substances in the blood.  Immunoassays included some commonly ordered lab tests such as tests to measure vitamin D or to detect prostrate cancer.  But many other routine blood tests, ranging from cholesterol to blood sugar, required completely different laboratory techniques.

Elizabeth needed a new device, one that could perform more than just one class of test.  In November 2010, she hired a young engineer named Kent Frankovich and put him in charge of designing it.

Kent had just earned a master’s degree in mechanical engineering from Stanford.  Prior to that, he’d worked for two years at NASA’s Jet Propulsion Laboratory in Pasadena, where he’d helped construct Curiosity, the Mars rover.  Kent recruited a friend—Greg Baney—from NASA to Theranos.

Carreyrou notes that for several months, Kent and Greg were Elizabeth’s favorite employees.  She joined their brainstorming sessions and offered some suggestions about what robotic systems they should consider.  Elizabeth called the new system the “miniLab.” 

Because the miniLab would be in people’s homes, it had to be small.

This posed engineering challenges because, in order to run all the tests she wanted, the miniLab would need to have many more components than the Edison.  In addition to Edison’s photomultiplier tube, the new device would need to cram three other laboratory instruments in one small space: a spectrophotometer, a cytometer, and an isothermal amplifier.

None of these were new inventions…

Laboratories all over the world had been using these instruments for decades.  In other words, Theranos wasn’t pioneering any new ways to test blood.  Rather, the miniLab’s value would lie in the miniaturization of existing lab technology.  While that might not amount to groundbreaking science, it made sense in the context of Elizabeth’s vision of taking blood testing out of central laboratories and bringing it to drugstores, supermarkets, and, eventually, people’s homes.

To be sure, there were already portable blood analyzers on the market.  One of them, a device that looked like a small ATM called the Piccolo Express, could perform thirty-one different blood tests and produce results in as little as twelve minutes.  It required only three or four drops of blood for a panel of a half dozen commonly ordered tests.  However, neither the Piccolo nor other existing portable analyzers could do the entire range of laboratory tests.  In Elizabeth’s mind, that was going to be the miniLab’s selling point.

Greg thought they should take off-the-shelf components and get the overall system working first before miniaturizing it.  Trying to miniaturize before having a working prototype didn’t make sense.  But Elizabeth wouldn’t hear of it.

In the spring of 2011, Elizabeth hired her younger brother, Christian.  Although two years out of college—Duke University—and with no clear qualifications to work at a blood diagnostics company, what mattered to Elizabeth was that she could trust her brother.  Christian soon recruited five fraternity brothers: Jeff Blickman, Nick Menchel, Dan Edlin, Sani Hadziahmetovic, and Max Fosque.  The became known inside Theranosas “the Frat Pack.”

Like Christian, none of the other Duke boys had any experience or training relevant to blood testing or medical devices, but their friendship with Elizabeth’s brother vaulted them above most other employees in the company hierarchy.

Meanwhile, Greg had brought several of his own friends, Jordan Carr, Ted Pasco, and Trey Howard.

Jordan, Trey, and Ted were all assigned to the product management group with Christian and his friends, but they weren’t granted the same level of access to sensitive information.  Many of the hush-hush meetings Elizabeth and Sunny held to strategize about the Walgreens and Safeway partnerships were off limits to them, whereas Christian and his fraternity brothers were invited in.

At the holiday party in December 2011, Elizabeth gave a speech which included the following:

“The miniLab is the most important thing humanity has ever built.  If you don’t believe this is the case, you should leave now.  Everyone needs to work as hard as humanly possible to deliver it.”

By this point, Greg had decided to leave Theranos in two months.  He had become disillusioned:

The miniLab Greg was helping build with a prototype, nothing more.  It needed to be tested thoroughly and fine-tuned, which would require time.  A lot of time.  Most companies went through three cycles of prototyping before they went to market with a product.  But Sunny was already placing orders for components to build one hundred miniLabs, based on a first, untested prototype.  It was as if Boeing built one plane and, without doing a single flight test, told airline passengers, “Hop aboard.”

One problem that would require a great deal of testing was thermal.  Packing many instruments together in a small space led to unpredicted variations in temperature.

 

THE WELLNESS PLAY       

Safeway’s business was struggling.  On an earnings call, CEO Steve Burd was asked why the company was buying back stock in order to boost earnings per share.  Burd responded that the company was about to do well, so buying back shares was the right move.  Burd elaborated by saying that the company was planning a significant “wellness play.” Analysts inferred that Safeway had a secret plan to ignite growth.

Burd had high hopes for the venture.  He’d ordered the remodeling of more than half of Safeway’s seventeen hundred stores to make room for upscale clinics with deluxe carpeting, custom wood cabinetry, granite countertops, and flat-screen TVs.  Per Theranos’s instructions, they were to be called wellness centers and had to look “better than a spa.”  Although Safeway was shouldering the entire cost of the $350 million renovation on its own, Burd expected it to more than pay for itself once the new clinics started offering the startup’s novel blood tests.

…[Burd] was starry-eyed about the young Stanford dropout and her revolutionary technology, which fit so perfectly with his passion for preventative healthcare.

Elizabeth had a direct line to Burd and answered only to him… He usually held his deputies and the company’s business partners to firm deadlines, but he allowed Elizabeth to miss one after the other.

In early 2012, the companies had agreed that Theranos would be in charge of blood testing at a Safeway employee health clinic on its corporate campus in Pleasanton.

Safeway’s first chief medical officer was Kent Bradley.  Bradley attended West Point and then the armed forces’ medical school in Bethesda, Maryland.  Then he served the U.S. Army for seventeen years before Safeway hired him.  Bradley looked forward to seeing the Theranos system in action.

However, he was surprised to learn that Theranos wasn’t planning on putting any of its devices in the Pleasanton clinic.  Instead, it had stationed two phlebotomists there to draw blood, and the samples they collected were couriered across San Francisco Bay to Palo Alto for testing.  He also noticed that the phlebotomists were drawing blood from every employee twice, once with a lancet applied to the index finger and a second time the old-fashioned way with a hypodermic needle inserted in the arm.  Why the need for venipunctures—the medical term for needle draws—if the Theranos finger-stick technology was fully developed and ready to be rolled out to consumers, he wondered.

Bradley’s suspicions were further aroused by the amount of time it took to get results back.  His understanding had been that the tests were supposed to be quasi-instantaneous, but some Safeway employees were having to wait as long as two weeks to receive their results.  And not every test was performed by Theranos itself.  Even though the startup had not said anything about outsourcing some of the testing, Bradley discovered that it was farming out some tests to a big reference laboratory in Salt Lake City called ARUP.

What really set off Bradley’s alarm bells, though, was when some otherwise healthy employees started coming to him with concerns about abnormal test results.  As a precaution, he sent them to get retested at a Quest or LabCorp location.  Each time, the new set of tests came back normal, suggesting the Theranos results were off…

Bradley put together a detailed analysis of the discrepancies.  Some of the differences between the Theranos values and the values from the other labs were disturbingly large.  When the Theranos values did match those of the other labs, they tended to be for tests performed by ARUP.

Bradley ended up taking his concerns to Burd, but Burd assured the doctor that Theranos’s technology had been tested and was reliable.

Theranos had a temporary lab in East Meadow Circle in Palo Alto.  The lab had gotten a certificate saying it was in compliance with CLIA—the federal law that governed clinical laboratories.  But such certificates were easy to obtain.

Although the ultimate enforcer of CLIA was the Centers for Medicare and Medicaid Services, the federal agency delegated most routine lab inspections to states.  In California, they were handled by the state department of health’s Laboratory Field Services division, which an audit had shown to be badly underfunded and struggling to fulfill its oversight responsibilities.

The East Meadows Circle lab didn’t contain a single Theranos proprietary device.  The miniLab was still being developed and was a long way from being ready for patient testing.  Instead, the lab had more than a dozen commercial blood and body-fluid analyzers made by companies such as Chicago-based Abbott Laboratories, Germany’s Siemens, and Italy’s DiaSorin.  Arne Gelb, a pathologist, ran the lab.  A handful of clinical laboratory scientists (CLSs) helped Arne.

One CLS named Kosal Lim was poorly trained and sloppy.  An experienced colleague, Diana Dupuy, believed Lim was harming the accuracy of the test results.

To Dupuy, Lim’s blunders were inexcusable. They included ignoring manufacturers’ instructions for how to handle reagents; putting expired reagents in the same refrigerator as current ones; running patient tests on lab equipment that hadn’t been calibrated; improperly performing quality-control runs on an analyzer; doing tasks he hadn’t been trained to do; and contaminating a bottle of Wright’s stain, a mixture of dyes used to differentiate blood cell types.

Dupuy documented Lim’s mistakes in regular emails to Arne and to Sunny, often including photos.

Dupuy also had concerns about the competence of the two phlebotomists Theranos had stationed in Pleasanton.  Blood is typically spun down in a centrifuge before it’s tested to separate its plasma from the patient’s blood cells.  The phlebotomists hadn’t been trained to use the centrifuge they’d been given and they didn’t know how long or at what speed to spin down patients’ blood.  When they arrived in Palo Alto, the plasma samples were often polluted with particulate matter.  She also discovered that many of the blood-drawing tubes Theranos was using were expired, making the anticoagulant in them ineffective and compromising the integrity of the specimens.

Dupuy was sent to Delaware to train on a new Siemens analyzer Theranos bought.  When she got back to the lab, it was spotless.

Sunny, who appeared to have been waiting for her, summoned her into a meeting room.  In an intimidating tone, he informed her that he had taken a tour of the lab in her absence and found not a single one of her complaints to be justified.

Sunny promptly fired her.  He rehired her based on Arne’s recommendation.  Then Sunny fired Dupuy again several weeks later.  She was immediately escorted from the building without a chance to grab her personal belongings.  Dupuy sent an email to Sunny—and copied Elizabeth—which included the following:

“I was warned by more than 5 people that you are a loose cannon and it all depends on your mood as to what will trigger you to explode.  I was also told that anytime someone deals with you it’s never a good outcome for that person.

The CLIA lab is in trouble with Kosal running the show and no one watching him or Arne.  You have a mediocre Lab Director taking up for a sub-par CLS for whatever reason.  I fully guarantee that Kosal will certainly make a huge mistake one day in the lab that will adversely affect patient results. I actually think he has already done this on several accounts but has put the blame on the reagents.  Just as you stated everything he touches is a disaster!

I only hope that somehow I bring awareness to you that you have created a work environment where people hide things from you out of fear.  You cannot run a company through fear and intimidation… it will only work for a period of time before it collapses.”

As for the Safeway partnership, Theranos kept pushing back the date for the launch.  Burd had to keep telling analysts and investors on each quarterly earnings call that the new program was just about to launch, only to have it be delayed again.  Safeway’s finance department forecast revenues of $250 million, which was aggressive.  The revenues hadn’t materialized, however, and Safeway had spent $350 million just to build the wellness centers.  Safeway’s board was starting to get upset.  Although Burd had done an excellent job during his first decade as CEO, the second decade hadn’t been very good.  The costs and delays associated with the wellness centers prompted the board to ask Burd to retire.  He agreed.

Safeway then had to contact Sunny or the Frat Pack if they wanted to communicate with Theranos.  Sunny always acted upset as if his time was too valuable, as if Theranos’s technology was a massive innovation requiring a huge time commitment.  Safeway executives were very upset about Sunny’s attitude.  But they still worried that they might miss out, so they didn’t walk away from the partnership.

 

“WHO IS LTC SHOEMAKER?”

Lieutenant Colonel David Shoemaker was part of a small military delegation meeting in Palo Alto in November 2011 to bless Theranos’s deploying its devices in the Afghan war theatre.  Only, instead of blessing the proposal, LTC Shoemaker told Elizabeth that there were various regulations her approach would fail to meet.

The idea of using Theranos devices on the battlefield had germinated the previous August when Elizabeth had met James Mattis, head of the U.S. Central Command, at the Marines’ Memorial Club in San Francisco.  Elizabeth’s impromptu pitch about how her novel way of testing blood from just a finger prick could help diagnose and treat wounded soldiers faster, and potentially save lives, had found a receptive audience in the four-star general.  Jim “Mad Dog” Mattis was fiercely protective of his troops, which made him one of the most popular commanders in the U.S. military.  The hard-charging general was open to pursuing any technology that might keep his men safer as they fought the Taliban in the interminable, atrocity-marred war in Afghanistan.

This type of request had to go through the army’s medical department.  Shoemaker’s job was to makes sure the army followed all laws and regulations when it tested medical devices.  With regard to Theranos, the company would have to get approval from the FDA at a minimum.

Elizabeth disagreed forcefully, citing advice Theranos received from its lawyers.  She was so defensive and obstinate that Shoemaker quickly realized that prolonging the argument would be a waste of time.  She clearly didn’t want to hear anything that contradicted her point of view.  As he looked around the table, he noted that she had brought no regulatory affairs expert to the meeting.  He suspected the company didn’t even employ one.  If he was right about that, it was an incredibly naïve way of operating.  Health care was the most highly regulated industry in the country, and for good reason: the lives of patients were at stake.

Soon thereafter, in response to an email from Shoemaker complaining to the FDA, Gary Yamamoto, a veteran field inspector in CMS’s regional office in San Francisco, was sent to exam Theranos’s lab.

[Elizabeth] and Sunny professed not to know what Shoemaker had been talking about in his email.  Yes, Elizabeth had met with the army officer, but she had never told him Theranos intended to deploy its blood-testing machines far and wide under the cover of a single CLIA certificate.

Yamamoto asked why Theranos had applied for a CLIA certificate.

Sunny responded that the company wanted to learn about how labs worked and what better way to do that than to operate one itself?  Yamamoto found that answer fishy and borderline nonsensical.  He asked to see their lab.

Carreyrou continues:

It looked like any other lab.  No sign of any special or novel blood-testing technology.  When he pointed this out, Sunny said the Theranos devices were still under development and the company had no plans to deploy them without FDA clearance—flatly contradicting what Elizabeth had told Shoemaker on not one but two occasions.  Yamamoto wasn’t sure what to believe.  Why would the army officer have made all that stuff up?

…If Theranos intended to eventually roll its devices out to other locations, those places would need CLIA certificates too. Either that or, better yet, the devices themselves would need to be approved by the FDA.

Elizabeth immediately sent an email to General Mattis accusing Shoemaker of giving “blatantly false information” to the FDA and CMS about Theranos.  Mattis was furious and wanted to get to the bottom of things ASAP.  A colleague of Shoemaker’s forwarded the emails, including Mattis’s responses, to Shoemaker.  Shoemaker got worried about what Mattis would do.

Shoemaker met with Mattis to answer the general’s questions.  Once Mattis learned more about the rules and regulations governing the situation—the medical devices couldn’t be tested on human subjects without FDA approval except under very strict conditions—he was reasonable.  In the meantime, they could conduct a “limited objective experiment” using leftover de-identified blood samples from soldiers.

Although Theranos had the green light to run the “limited objective experiment,” for some reason it never proceeded to do so.

 

LIGHTING A FUISZ

On October 29, 2011, Richard Fuisz was served a set of papers.  It was a lawsuit filed by Theranos in federal court in San Francisco alleging that Fuisz had conspired with his sons from his first marriage, Joe and John Fuisz, to steal confidential patent information to develop a rival patent.  The suit alleged that the theft had been done by John Fuisz while he was employed at Theranos’s former patent counsel, McDermott Will & Emery.

Fuisz and his sons were angered by the suit, but they weren’t overly worried about it at first.  They were confident in the knowledge that its allegations were false.

Carreyrou writes:

John had no reason to wish Elizabeth or her family ill; on the contrary.  When he was in his early twenties, Chris Holmes had written him a letter of recommendation that helped him gain admission to Catholic University’s law school.  Later, John’s first wife had gotten to know Noel Holmes through Lorraine Fuisz and become friendly with her.  Noel had even dropped by their house when John’s first son was born to bring the baby a gift.

Moreover, Richard and John Fuisz weren’t close.  John thought his father was an overbearing megalomaniac and tried to keep their interactions to a bare minimum.  In 2004, he’d even dropped him as a McDermott client because he was being difficult and slow to pay his bills.  The notion that John had willingly jeopardized his legal career to steal information for his father betrayed a fundamental misunderstanding of their frosty relationship.

But Elizabeth was understandably furious at Richard Fuisz.  The patent application he had filed in April 2006 had matured into U.S. Patent 7,824,612 in November 2010 and now stood in the way of her vision of putting the Theranos device in people’s homes.  If that vision was someday realized, she would have to license the bar code mechanism Fuisz had thought up to alert doctors to patients’ abnormal blood-test results.  Fuisz had rubbed that fact in her face the day his patent was issued by sending a Fuisz pharma press release to info@theranos.com, the email address the company provided on its website for general queries.  Rather than give in to what she saw as blackmail, Elizabeth had decided to steamroll her old neighbor by hiring one of the country’s best and most feared attorneys to go after him.

The Justice Department had hired David Boies to handle its antitrust suit against Microsoft.  Boies won a resounding victory in that case, which helped him rise to national prominence.  Just before Theranos sued, all three Fuiszes—Richard, John, and Joe—could tell that they were under surveillance by private investigators.

Boies’s use of private investigators wasn’t an intimidation tactic, it was the product of a singular paranoia that shaped Elizabeth and Sunny’s view of the world.  That paranoia centered on the belief that the lab industry’s two dominant players, Quest Diagnostics and Laboratory Corporation of America, would stop at nothing to undermine Theranos and its technology.  When Boies had first been approached about representing Theranos by Larry Ellison and another investor, it was that overarching concern that had been communicated to him.  In other words, Boies’s assignment wasn’t just to sue Fuisz, it was to investigate whether he was in league with Quest and LabCorp.  The reality was that Theranos was on neither company’s radar at that stage and that, as colorful and filled with intrigue as Fuisz’s life had been, he had no connection to them whatsoever.

Boies had zero evidence that John Fuisz had done what Theranos alleged.  Nonetheless, Boies intended to use several things from John’s past to create doubt in a judge or jury.  Potentially the most damaging thing Boies wanted to use was that McDermott had made John resign in 2009 after he had an argument with the firm’s other partners.  John insisted the firm discontinue its reliance on a forged document in a case before the International Trade Commission in which McDermott was representing a Chinese state-owned company against the U.S. government’s Office of Unfair Import Investigations.  McDermott leaders agreed to withdraw the document, but that decision significantly weakened the Chinese client’s defense.  Senior partners got upset about it.  They argued that there had been several incidents when John didn’t behave as a partner should.  One incident was a complaint a client had made—this was Elizabeth’s September 2008 complaint about Richard Fuisz’s patent.

Eventually John was beyond furious.  He had launched his own practice after leaving McDermott.  The Theranos allegations had caused him to lose several clients.  Moreover, opposing counsel mentioned the allegations in order to tar John.  Finally, his wife had been diagnosed with vasa previa—a pregnancy complication where the fetus’s blood vessels are dangerously exposed.  This added to John’s stress.

John always had a short fuse.  During the deposition by Boies’s partners, John was combative and ornery.  He used foul language while threatening to harass Elizabeth “till she dies, absolutely.”

In the meantime, Richard and Joe Fuisz were worrying about how expensive the litigation was getting.  Also, they knew they were up against one of the most expensive lawyers in the world: David Boies, who was reported to make $10 million a year.  But they didn’t know that Boies had agreed to accept stock in Theranos in place of his usual fees.  Partly out of concern for his investment, Boies began attending all of the company’s board meetings in early 2013.

Richard Fuisz examined Theranos’s patents.  He noticed that the name Ian Gibbons often appeared.  Gibbons was British and had a Ph.D. in biochemistry from Cambridge.  Fuisz suspected that Gibbons and other Theranos employees with advanced degrees had done most of the technical work related to Theranos’s patents.

 

IAN GIBBONS

Elizabeth hired Ian Gibbons on the recommendation of her Stanford mentor, Channing Robertson.

Ian fit the stereotype of the nerdy scientists to a T.  He wore a beard and glasses and hiked his pants way above his waist.  He could spend hours on end analyzing data and took copious notes documenting everything he did at work.  This meticulousness carried over to his leisure time: he was an avid reader and kept a list of every single book he’d read.  It included Marcel Proust’s seven-volume opus, Remembrance of Things Past, which he reread more than once.

Ian met his wife Rochelle at Berkeley in the 1970s.  He was doing a postdoctorate fellowship in molecular biology, while Rochelle was doing graduate research.  They didn’t have children.  But they loved their dogs Chloe and Lucy, and their cat Livia, named after the wife of the Roman emperor Augustus.  Ian also enjoyed going to the opera and photography.  He altered photos for fun.

Ian’s specialty was immunoassays.  He was passionate about the science of bloodtesting.  He also enjoyed teaching it.  Early on at Theranos, he would give small lectures to the rest of the staff.

Ian insisted that the blood tests they designed be as accurate in Theranos devices as they were on the lab bench.  Because this was rarely the case, Ian was quite frustrated.

He and Tony Nugent butted heads over this issue during the development of the Edison. As admirable as Ian’s exacting standards were, Tony felt that all he did was complain and that he never offered any solutions.

Ian also had issues with Elizabeth’s management, especially the way she siloed the groups off from one another and discouraged them from communicating.  The reason she and Sunny invoked for this way of operating was that Theranos was “in stealth mode,” but it made no sense to Ian.  At the other diagnostics companies where he had worked, there had always been cross-functional teams with representatives from the chemistry, engineering, manufacturing, quality control, and regulatory departments working toward a common objective.  That was how you got everyone on the same page, solved problems, and met deadlines.

Elizabeth’s loose relationship with the truth was another point of contention.  Ian had heard her tell outright lies more than once and, after five years of working with her, he no longer trusted anything she said, especially when she made representations to employees or outsiders about the readiness of the company’s technology.

Ian complained confidentially to his friend Channing Robertson.  But Robertson turned around and told Elizabeth all that Ian said.  Elizabeth fired him.  Sunny called the next day because several colleagues urged Elizabeth to reconsider.  Ian was brought back but he was no longer head of general chemistry.  Instead, he was a technical consultant. 

Ian wasn’t the only employee being sidelined at that point.  It seemed that the old guard was being mothballed in favor of new recruits.  Nonetheless, Ian took it hard.

One day, Tony and Ian—who’d both been marginalized—got to talking.  Tony suggested that perhaps the company was merely a vehicle for Elizabeth and Sunny’s romance and that none of the work they didn’t actually mattered.  Ian agreed, saying, “It’s a folie a deux.”  Tony looked up the definition of that expression, which seemed accurate to him: “The presence of the same or similar delusional ideas in two persons closely associated with one another.”

Ian kept working closely with Paul Patel, who had replaced Ian.  Paul had enormous respect for Ian and continued treating him as an equal, consulting him on everything.  However, Paul avoided conflict and was more willing than Ian to compromise with the engineers while building the miniLab.  Ian wouldn’t compromise and got upset.  Paul frequently had to calm him down over the phone at night. Ian told Paul to abide by his convictions and never lose sight of concern for the patient.

Sunny put Samartha Anekal, who had a Ph.D. in chemical engineering, in charge of integrating the parts of the miniLab.  Sam struck some as a yes-man who simply did what Sunny told him to do.

As these things were unfolding, Ian had gotten clinically depressed—except he hadn’t been diagnosed as such.  He started drinking heavily in the evenings.  Rochelle was grieving for her mother, who had just passed away, and didn’t notice how depressed Ian was getting.

Theranos told Ian he’d been subpoenaed to testify in the Fuisz case.  Because Rochelle had done work as a patent attorney, Ian asked her to look the Theranos’s patents. 

While doing so, she noticed that Elizabreth’s name was on all the company’s patents, often in first place in the list of inventors.  When Ian told her that Elizabeth’s scientific contribution had been negligible, Rochelle warned him that the patents could be invalidated if this was ever exposed.  That only served to make him more agitated.

On May 15, he called to set up a meeting with Elizabeth.  After an appointment was set for the next day, Ian started worrying that Elizabeth would fire him.  The same day, the Theranos lawyer David Doyle told Ian that Fuizs’s lawyers—after trying for weeks to get the Boies Schiller attorneys to propose a date for Ian’s deposition—required Ian to appear at their offices in Campbell, California, on May 17.

The morning of May 16, Ian’s wife discovered that he’d taken enough acetaminophen to kill a horse.  He was pronounced dead on May 23.  Theranos had virtually no response. 

Although Tony Nugent and Ian had fought all the time, Tony felt bad about the lack of empathy for someone who had given a decade of his life to the company.  Tony downloaded a list of Ian’s patents and created an email, including a photo of Ian, which Tony sent to two dozen colleagues who’d worked with him.

 

CHIAT/DAY                                         

Chiat/Day was working on a secret marketing campaign for Theranos.  Patrick O’Neill, the creative director of the company’s Los Angeles office, was in charge.

Elizabeth had chosen Chiat/Day because it was the agency that represented Apple for many years, creating its iconic 1984 Macintosh ad and later its “Think Different” campaign in the late 1990s.  She’d even tried to convince Lee Clow, the creative genius behind those ads, to come out of retirement to work for her.  Clow politely referred her back to the agency, where she had immediately connected with Patrick.

Patrick was drawn in by Elizabeth’s extreme determination to do something great.  The Theranos mission to give people pain-free, low-cost health care was inspiring.  Advertisers don’t often get a chance to work on something that can make the world better, observes Carreyrou.

Part of the campaign included pictures of patients—played by models—of all different ages, genders, and ethnicities. 

The message was that Theranos’s blood-testing technology would help everyone.

Carreyrou again:

Real blood tended to turn purple after awhile when it was exposed to air, so they filled one of the nanotainters with fake Halloween blood and took pictures of it against a white background.  Patrick then made a photo montage showing it balancing on the tip of a finger.  As he’d anticipated, it made for an arresting visual.  Mike Yagi tried out different slogans to go with it, eventually settling on two that Elizabeth really liked: “One tiny drop changes everything” and “The lab test, reinvented.”…

Patrick also worked with Elizabeth on a new company logo.  Elizabeth believed in the Flower of Life, a geometric pattern of intersecting circles within a larger circle that pagans once considered the visual expression of the life that runs through all sentient beings.

Although Patrick was enthused, his colleague Stan Fiorito was more circumspect.  He thought something about Sunny was strange.  He kept using software engineering jargon in weekly meetings even though it had zero applicability to the marketing discussions.  Also, Theranos was paying Chiat/Day $6 million a year.  Where was it getting the money for this?  Elizabeth stated several times that the army was using Theranos technology on the battlefield in Afghanistan.  She claimed it was saving soldiers’ lives.  Perhaps Theranos was funded by the Pentagon, thought Stan.  At least that would help explain the extreme secrecy the company insisted upon.

Besides Mike Yagi, Stan supervised Kate Wolff and Mike Pedito.  Kate and Mike were no-nonsense people and they began to wonder about Theranos.

Elizabeth wanted the website and all the various marketing materials to feature bold, affirmative statements.  One was that Theranos could run “over 800 tests”on a drop of blood.  Another was that its technology was more accurate than traditional lab testing.  She also wanted to say that Theranos test results were ready in less than thirty minutes and that its tests were “approved by FDA” and “endorsed by key medical centers” such as the Mayo Clinic and the University of California, San Francisco’s medical school, using the FDA, Mayo Clinic, and UCSF logos.

When she inquired about the basis for the claim about Theranos’s superior accuracy, Kate learned that it was extrapolated from a study that had concluded that 93 percent of lab mistakes were due to human error.  Theranos argued that, since its testing process was fully automated inside its device, that was grounds enough to say that it was more accurate than other labs.  Kate thought that was a big leap in logic and said so.  After all, there were laws against misleading advertising.

Mike agreed with Kate.

Elizabeth had mentioned a report several hundred pages long supporting Theranos’s scientific claims.  Kate and Mike repeatedly asked to see it, but Theranos wouldn’t produce it.  Instead, the company sent them a password-protected file containing what it said were excerpts from the report.  It stated that the Johns Hopkins University School of Medicine had conducted due diligence on the Theranos technology and found it “novel and sound” and capable of “accurately” running “a wide range of routine and special assays.”

Those quotes weren’t from any lengthy report, however.  They were from the two-page summary of Elizabeth and Sunny’s meeting with five Hopkins officials in April 2010.  As it had done with Walgreens, Theranos was again using that meeting to claim that its system had been independently evaluated.  But that simply wasn’t true.  Bill Clarke, the director of clinical toxicology at the Johns Hopkins Hospital and one of three university scientists who attended the 2010 meeting, had asked Elizabeth to ship one of her devices to his lab so he could put it through its paces and compare its performance to the equipment he normally used.  She had indicated she would but had never followed through.  Kate and Mike didn’t know any of this, but the fact that Theranos refused to show them the full report made them suspicious.

To learn how to market to doctors, Chiat/Day suggested doing focus group interviews with a few physicians.  Theranos agreed as long as it was very secret.  Kate asked her wife, Tracy, chief resident at Los Angeles County General, to participate.  Tracy agreed.  During a phone interview, Tracy asked a few questions that no one at Theranos seemed to be able to answer.  Tracy told Kate that she doubted the company had any new technology.  She also doubted you could get enough blood from a finger to run tests accurately.

The evening before the marketing campaign was going to launch, Elizabeth set up an emergency conference call.  She systematically dialed back the language that would be used.  “Welcome to a revolution in lab testing” was changed to “Welcome to Theranos.”  “Faster results.  Faster answers.” became “Fast results.  Fast answers.”  “A tiny drop is all it takes” was now “A few drops is all it takes.”  “Goodbye, big bad needle” (which referred only to finger-stick draws) was replaced with “Instead of a huge needle, we can use a tiny finger stick or collect a micro-sample from a venous draw.”

Not everyone at Chiat/Day was skeptical, however.  Patrick thought Theranos could become his own big legacy, just as Apple had been for Lee Clow.

 

GOING LIVE

Alan Beam decided to become a doctor because his conservative Jewish parents thought that only law, medicine, or business was an appropriate career choice.  While attending Mount Sinai’s School of Medicine, he didn’t like the crazy hours or the sights and smells of the hospital ward.  Instead, he got interested in laboratory science.  He pursued postdoctoral studies in virology and a residency in clinical pathology at Brigham and Women’s Hospital in Boston.

In the summer of 2012, having recently read Walter Isaacson’s biography of Steve Jobs—which greatly inspired Alan—he wanted to move to the San Francisco Bay Area.  He ended up being hired as laboratory directory as Theranos.  He didn’t start until April 2013 because it took eight months before he got his California medical license.

After starting, Alan became concerned about low morale in the lab:

Its members were downright despondent.  During Alan’s first week on the job, Sunny summarily fired one of the CLSs.  The poor fellow was frog-marched out by security in front of everyone.  Alan got the distinct impression it wasn’t the first time something like that had happened.  No wonder people’s spirits were low, he thought.

The lab Alan inherited was divided into two parts: a room on the building’s second floor that was filled with commercial diagnostic equipment, and a second room beneath it where research was being conducted.  The upstairs room was the CLIA-certified part of the lab, the one Alan was responsible for.  Sunny and Elizabeth viewed its conventional machines as dinosaurs that would soon be rendered extinct by Theranos’s revolutionary technology, so they called it “Jurassic Park.”  They called the downstairs room “Normandy” in reference to the D-day landings during during World War II.  The proprietary Theranos devices it contained would take the lab industry by storm, like the Allied troops who braved hails of machine-gun fire on Normandy’s beaches to liberate Europe from Nazi occupation.

Alan liked the bravado at first.  But then he learned from Paul Patel—the biochemist leading the development of blood tests for Theranos’s new device (now called the “4S” instead of the miniLab)—that he and his team were still developing its assays on lab plates on the bench.  Alan asked Paul about it and Paul said the new Theranos box wasn’t working.

By the summer of 2013, the 4S had been under development for more than two and a half years.  But it still had a long list of problems.  Carreyrou writes:

The biggest problem of all was the dysfunctional corporate culture in which it was being developed.  Elizabeth and Sunny regarded anyone who raised a concern or an objection as a cynic and a naysayer.  Employees who persisted in doing so were usually marginalized or fired, while sycophants were promoted.  Sunny had elevated a group of ingratiating Indians to key positions…

For the dozens of Indians Theranos employed, the fear of being fired was more than just the dread of losing a paycheck.  Most were on H-1B visas and dependent on their continued employment at the company to remain in the country.  With a despotic boss like Sunny holding their fates in his hands, it was akin to indentured servitude.  Sunny, in fact, had the master-servant mentality common among an older generation of Indian businessmen.  Employees were his minions.  He expected them to be at his disposal at all hours of the day or night and on weekends. He checked the security logs every morning to see when they badged in and out…

With time, some employees grew less afraid of him and devised ways to manage him, as it dawned on them that they were dealing with an erratic man-child of limited intellect and an even more limited attention span.

Some of the problems were because Elizabeth was fixated on certain things.  For instance, she thought the 4S—aka the miniLab—was a consumer device like an iPhone, and therefore it had to be small and pretty.  She still hoped these devices would be in people’s homes someday.

Another difficulty stemmed from Elizabeth’s insistence that the miniLab be capable of performing the four major classes of blood tests: immunoassays, general chemistry assays, hematology assays, and assays that relied on the amplification of DNA.  The only known approach that would permit combining all of them in one desktop machine was to use robots wielding pipettes.  But this approach had an inherent flaw: overtime, a pipette’s accuracy drifts… While pipette drift was something that ailed all blood analyzers that relied on pipetting systems, the phenomenon was particularly pronounced on the miniLab. Its pipettes had to be recalibrated every two to three months, and the recalibration process put the device out of commission for five days.

Another serious weakness of the miniLab was that it could process only one blood sample at a time.  Commercial machines—which were bulky—could process hundreds of samples at the same time.

If the Theranos wellness centers attracted a lot of patients, the miniLab’s low throughput would cause long wait times, which was clearly inconsistent with the company’s promise of fast test results.

Someone suggested putting six miniLabs on top of one another—sharing one cytometer.  They adopted a computer term to name it: the“six-blade.”  But they overlooked a basic issue: temperature.  Some types of blood test require a very specific temperature. Because heat rises, the miniLabs near the top wouldn’t function.

There were other problems, too.  Many of them were fixable but would require a relatively long time.  Carreyrou explains:

Less than three years was not a lot of time to develop and perfect a complex medical device… The company was still several years away from having a viable product that could be used on patients.

However, as Elizabeth saw it, she didn’t have several years.  Twelve months earlier, on June 5, 2012, she’d designed a new contract with Walgreens that committed Theranos to launch its blood-testing services in some of the chain’s stores by February 1, 2013, in exchange for a $100 million “innovation fee” and an additional $40 million loan.

Theranos had missed that deadline—another postponement in what from Walgreens’s perspective had been three years of delays.  With Steve Burd’s retirement, the Safeway partnership was already falling apart, and if she waited much longer, Elizabeth risked losing Walgreens too. She was determined to launch in Walgreens stores by September, come hell or high water.

Since the miniLab was in no state to be deployed, Elizabeth and Sunny decided to dust off the Edison and launch with the older device.  That, in turn, led to another fateful decision—the decision to cheat.

Daniel Young, head of Theranos’s biomath team, and Xinwei Gong (who went by Sam), told Alan Beam that he and Sam were going to tinker with the ADVIA, one of the lab’s commercial analyzers.  It weighed 1,320 pounds and was made by Siemens Healthcare.  Since the Edison could only do immunoassays, Alan grasped why Daniel and Sam were going to try to use the ADVIA, which specialized in general chemistry assays.  As Carreyrou describes it:

One of the panels of blood tests most commonly ordered by physicians was known as the “chem 18” panel.  Its components, which ranged from tests to measure electrolytes sodium, potassium, and chloride to tests used to monitor patients’ kidney and liver function, were all general chemistry assays.  Launching in Walgreens stores with a menu of blood tests that didn’t include these tests would have been pointless.  They accounted for about two-thirds of doctors’ orders.

But the ADVIA was designed to handle a larger quantity of blood than you could obtain by pricking a finger.  So Daniel and Sam thought up a series of steps to adapt the Siemens analyzer to smaller samples.  Chief among these was the use of a big robotic liquid handler called the Tecan to dilute the little blood samples collected in the nanotainters with a saline solution.  Another was to transfer the diluted blood into custom-designed cups half the size of the ones that normally went into the ADVIA.

Because they were working with small blood samples, Daniel and Sam concluded that they would have to dilute the blood not once, but twice.  Alan knew this was a bad idea:

Any lab director worth his salt knew that the more you tampered with a blood sample, the more room you introduced for error.

Moreover, this double dilution lowered the concentration of the analytes in the blood samples to levels that were below the ADVIA’s FDA-sanctioned analytic measurement range. In other words, it meant using the machine in a way that neither the manufacturer nor its regulator approved of.  To get the final patient result, one had to multiply the diluted result by the same factor the blood had been diluted by, not knowing whether the diluted result was even reliable.  Daniel and Sam were nonetheless proud of what they’d accomplished.  At heart, both were engineers for whom patient care was an abstract concept.  If their tinkering turned out to have adverse consequences, they weren’t the ones who would be held personally responsible. It was Alan’s name, not theirs, that was on the CLIA certificate.

Anjali Laghari was in charge of the immunoassay group.  She’d worked with Ian Gibbons for a decade.  Anjali had spent years trying to get the Edison working, but the device still had a high error rate.

When Anjali started hearing that Theranos was “going live,” she grew very concerned.  She emailed Elizabeth and Daniel Young to remind them about the high error rates for some blood tests run on the Edison.

Neither Elizabeth nor Daniel acknowledged her email.  After eight years at the company, Anjali felt she was at an ethical crossroads.  To still be working out the kinks in the product was one thing when you were in R&D mode and testing blood volunteered by employees and their family members, but going live in Walgreens stores meant exposing the general population to what was essentially a big unauthorized research experiment.  That was something she couldn’t live with.  She decided to resign.

Elizabeth wanted to persuade Anjali to stay.  Anjali asked Elizabeth: Why should they rush to launch before their technology was ready?  

“Because when I promise something to a customer, I deliver.”

Anjali questioned this line of thought.  The customers who really mattered were the patients who ordered blood tests, believing that the tests were a reliable basis for medical decisions.

After Anjali resigned, her deputy Tina Noyes resigned.

The resignations infuriated Elizabeth and Sunny.  The following day they summoned the staff for an all-hands meeting in the cafeteria… Still visibly angry, Elizabeth told the gathered employees that she was building a religion.  If there were any among them who didn’t believe, they should leave.  Sunny put it more blatantly: anyone not prepared to show complete devotion and unmitigated loyalty to the company should “get the fuck out.”

 

UNICORN

Elizabeth had met the great statesman George Shultz a couple of years before 2013.  She impressed him and won his support.  Based on this connection, Elizabeth had been able to engineer a very favorable piece in the Wall Street Journal.  The article was published September 7, 2013, just as Theranos was going to launch its blood-testing services.  Carreyrou says of the article:

Drawing blood the traditional way with a needle in the arm was likened to vampirism… Theranos’s processes, by contrast,were described as requiring “only microscopic blood volumes” and as “faster, cheaper, and more accurate than the conventional methods.”  The brilliant young Stanford dropout behind the breakthrough invention was anointed “the next Steve Jobs or Bill Gates” by no less than former secretary of state George Shultz, the man many credited with winning the cold war, in a quote at the end of the article.

Elizabeth planned to use the misleading article and the Walgreens launch as a basis for a new fundraising campaign.

Donald A. Lucas, son of legendary venture capitalist Donald L. Lucas, called Mike Barsanti.  Don and Mike had been friendly since they attended Santa Clara University in the 1980s.  Don proceeded to pitch Mike on Theranos.

Mike had first heard about Elizabeth seven years earlier.  Mike had been interested then, but Don hadn’t been. 

[Back in 2006, Mike] asked Don why the firm wasn’t taking a flyer on [Elizabeth] like his father had.  Don had replied that after careful consideration he’s decided against it.  Elizabeth was all over the place, she wasn’t focused, his father couldn’t control her even though he chaired her board, and Don didn’t like or trust her, Mike recalled his friend telling him.

In 2013, Mike asked Don what had changed.

Don explained excitedly that Theranos had come a long way since then.  The company was about to announce the launch of its innovative finger-stick tests in one of the country’s largest retail chains.  And that wasn’t all, he said.  The Theranos devices were also being used by the U.S. military.

“Did you know they’re in the back of Humvees in Iraq?” he asked Mike.

[…]

If all this were true, they were impressive developments, Mike thought.

…Intent on seizing what he saw as a great opportunity, the Lucas Venture Group was raising money for two new funds, Don told Mike.  One of them was a traditional venture fund that would invest in several companies, including Theranos.  The second would be exclusively devoted to Theranos.  Did Mike want in?  If so, time was short.

Mike got an email on September 9, 2013, discussing the “Theranos-time sensitive” opportunity.  The Lucas Venture group would get a discounted price, which valued the firm at $6 billion.  Don mentioned that Theranos had “signed contracts and partnerships with very large retailers and drug stores as well as various pharmaceutical companies, HMO’s, insurance agencies, hospitals, clinics, and various government agencies.”  Don also said that the company had been “cash flow positive since 2006.”

Theranos seemed to be another “unicorn.”  Unicorns like Uber had been able to raise massive amounts of money while still remaining private companies, allowing them to avoid the pressures and scrutiny of going public.

Christopher James and Brian Grossman ran the hedge fund Partner Fund Management, which had $4 billion under management.  James and Grossman saw the Wall Street Journal article about Theranos and were interested.  They reached out to Elizabeth and went to meet with her on December 15, 2013.

During that first meeting, Elizabeth and Sunny told their guests that Theranos’s proprietary finger-stick technology could perform blood tests covering 1,000 of the 1,300 codes laboratories used to bill Medicare and private health insurers, according to a lawsuit Partner Fund later filed against the company.  (Many blood tests involve several billing codes, so the actual number of tests represented by those thousand codes was in the low hundreds.)

At a second meeting three weeks later, they showed them a Powerpoint presentation containing scatter plots purporting to compare test data from Theranos’s proprietary analyzers to data from conventional lab machines.  Each plot showed data points tightly clustered around a straight line that rose up diagonally from the horizontal x-axis.  This indicated that Theranos’s tests results were almost perfectly correlated with those of the conventional machines.  In other words, its technology was as accurate as traditional testing.  The rub was that much of the data in the charts wasn’t from the miniLab or even from the Edison.  It was from other commercial blood analyzers Theranos had purchased, including one manufactured by a company located an hour north of Palo Alto called Bio-Rad.

Sunny also told James and Grossman that Theranos had developed about three hundred different blood tests, ranging from commonly ordered tests to measure glucose, electrolytes, and kidney function to more esoteric cancer-detection tests.  He boasted that Theranos could perform 98 percent of them on tiny blood samples pricked from a finger and that, within six months, it would be able to do all of them that way.  These three hundred tests represented 99 to 99.9 percent of all laboratory requests, and Theranos had submitted every single one of them to the FDA for approval, he said.

Sunny and Elizabeth’s boldest claim was that the Theranos system was capable of running seventy different blood tests simultaneously on a single finger-stick sample and that it would soon be able to run even more.  The ability to perform so many tests on just a drop or two of blood was something of a Holy Grail in the field of microfluidics.

There were some basic problems with trying to run many tests on small samples of blood.  If you used a micro blood sample to do an immunoassay, then there usually wasn’t enough blood for the different set of lab techniques a general chemistry or hematology assay required.  Another fundamental problem was that in transferring a small sample to a microfluidic chip, some blood was lost.  This doesn’t matter for large blood samples, but it can be a crucial problem for small blood samples.  Yet Elizabeth and Sunny implied that they had solved these and other difficulties.

James and Grossman not only liked the presentations by Elizabeth and Sunny; they also were impressed by Theranos’s board of directors.  In addition to Shultz and General Mattis, the board now had Henry Kissinger, William Perry (former secretary of defense), Sam Nunn, and former navy admiral Gary Roughead.  Like Shultz, all of these board members were fellows at the Hoover Institution at Stanford.

Sunny sent the hedge fund managers a spreadsheet with financial projections.

It forecast gross profits of $165 million on revenues of $261 million in 2014 and gross profits of $1.08 billion on revenues of $1.68 billion in 2015.  Little did they know that Sunny had fabricated these numbers from whole cloth.  Theranos hadn’t had a real chief financial officer since Elizabeth had fired Henry Mosley in 2006.

Partner Fund invested $96 million.  This valued Theranos at $9 billion, which put Elizabeth’s net worth at almost $5 billion.

 

THE GRANDSON       

Carreyrou writes this chapter about Tyler Shultz, the grandon of George Shultz:

Tyler had first met Elizabeth in late 2011 when he’d dropped by his grandfather George’s house near the Stanford campus.  He was a junior then, majoring in mechanical engineering.  Elizabeth’s vision of instant and painless tests run on drops of blood collected from fingertips had struck an immediate chord with him.  After interning at Theranos that summer, he’d changed his major to biology and applied for a full-time position at the company.

Tyler became friends with Erika Cheung.

Their job on the immunoassay team was to help run experiments to verify the accuracy of blood tests on Theranos’s Edison devices before they were deployed in the lab for use on patients.  This verification process was known as “assay validation.”

[…]

One type of experiment he and Erika were tasked with doing involved retesting blood samples on the Edisons over and over to measure how much their results varied. The data collected were used to calculate each Edison’s blood test’s coefficient of variation, or CV.  A test is generally considered precise if its CV is less than 10 percent.  To Tyler’s dismay, data runs that didn’t achieve low enough CVs were simply discarded and the experiments repeated until the desired number was reached.  It was as if you flipped a coin enough times to get ten heads in a row and then declared that the coin always returned heads.  Even with the “good” data runs, Tyler and Erika noticed that some values were deemed outliers and deleted.  When Erika asked the group’s more senior scientists how they defined an outlier, no one could give her a straight answer.  Erika and Tyler might be young and inexperienced, but they both knew that cherry-picking data wasn’t good science.  Nor were they the only ones who had concerns about these practices.

Tyler and colleagues tested 247 blood samples on Edison for syphilis, 66 of which were known to be positive.  The devices correctly identified only 65 percent of the sample on the first run, and 80 percent on the second run.

Yet, in its validation report, Theranos stated that its syphilis test had a sensitivity of 95 percent.

There were other tests where Tyler and Erika thought Theranos was being misleading.  For instance, a blood sample would be tested for vitamin D on an analyzer made by the Italian company DiaSorin.  It might show a vitamin D concentration of 20 nanograms per milliliter—a normal result for a healthy patient.  When Erika tested the sample on the Edison, the result was 10 or 20 nanograms per milliliter—indicating a vitamin D deficiency.  Nonetheless, the Edison was cleared for use in the clinical lab on live patient samples, writes Carreyrou.

In November 2013, while working in the clinical lab, Erika received a patient order from the Walgreens store in Palo Alto.  As was standard practice, first she did a quality-control check.  That involves testing a sample where you already know the concentration of the analyte.

If the result obtained is two standard deviations higher or lower than the known value, the quality-control check is usually deemed to have failed.

Erik’a first quality-control check failed.  She ran it again and that one failed as well.  Because it was during Thanksgiving, no one Erika normally reported to was around.  Erika sent an email to the company’s emergency help line.

Sam Anekal, Suraj Saksena, and Daniel Young responded to her email with various suggestions, but nothing they proposed worked.  After awhile an employee named Uyen Do from the research-and-development side came down and took a look at the quality-control readings.

Twelve values had been generated, six during each quality-control test.

Without bothering to explain her rationale to Erika, Do deleted two of those twelve values, declaring them outliers.  She then went ahead and tested the patient sample and sent out a result.

This wasn’t how you were supposed to handle repeat quality-control failures.  Normally, two such failures in a row would have been cause to take the devices off-line and recalibrate them.  Moreover, Do wasn’t even authorized to be in the clinical lab.  Unlike Erika, she didn’t have a CLS license and had no standing to process patient samples.  The episode left Erika shaken.

Tyler Shultz moved to the production team in early 2014.  This put him back near Erika and other colleagues from the clinical lab.

Tyler learned from Erika and others that the Edisons were frequently flunking quality-control checks and that Sunny was pressuring lab personnel to ignore the failures and to test patient samples on the devices anyway.

Tyler asked Elizabeth about validation reports, and she suggested he speak with Daniel Young.  Tyler asked Daniel about CV values: Why were so many data runs discarded when the resulting CV was too high?   Daniel told him that he was making the mistake of taking into account all six values generated by the Edison during a test.  Young said that only the median value mattered.  It was obvious to Tyler that if the Edison’s results were accurate, such data contortions—and the associated dishonesty—wouldn’t be needed in the first place.

Furthermore, all clinical laboratories undergo “proficiency testing” three times a year.

During its first two years of operation, the Theranos lab had always tested proficiency-testing samples on commercial analyzers.  But since it was now using the Edisons for some patient tests, Alan Beam and his new lab codirector had been curious to see how the devices fared in the exercise.  Beam and the new codirector, Mark Pandori, had ordered Erika and other lab associates to split the proficiency-testing samples and run one part on the Edisons and the other part on the lab’s Siemens and DiaSorin analyzers for comparison.  The Edison results had differed markedly from the Siemens and DiaSorin ones, especially for vitamin D.

When Sunny had learned of their little experiment, he’d hit the roof.  Not only had he put an immediate end to it, he had made them report only the Siemens and DiaSorin results.  There was a lot of chatter in the lab that the Edison results should have been the ones reported.  Tyler had looked up the CLIA regulations and they seemed to bear that out…

Tyler told Daniel he didn’t see how what Theranos had done could be legal.  Daniel’s response followed a tortuous logic.  He said a laboratory’s proficiency-testing results were assessed by comparing them to its peers’ results, which wasn’t possible in Theranos’s case because its technology was unique and had no peer group.  As a result, the only way to do an apples-to-apples comparison was by using the same conventional methods as other laboratories.  Besides, proficiency-testing rules were extremely complicated, he argued.  Tyler could rest assured that no laws had been broken.  Tyler didn’t buy it.

In March 2014, using an alias, Tyler emailed the New York health department because it ran one of the proficiency-testing programs in which Theranos had participated.  Without revealing the name of the company in question, he asked about Theranos’s approach.  He got confirmation that Theranos’s practices were “a form of PT cheating” and were “in violation of the state and federal requirements.”  Tyler was given a choice: reveal the name of the company or file an anonymous complaint with New York State’s Laboratory Investigative Unit.  He chose the second option.

Tyler told his famous grandfather George about his concerns.  He said, moreover, that he was going to resign.  George asked him to give Elizabeth a chance to respond.  Tyler agreed.  Elizabeth was too busy to meet in person, so Tyler sent her a detailed email.  He didn’t hear anything for a few days.

When the response finally arrived, it didn’t come from Elizabeth.  It came from Sunny.  And it was withering.  In a point-by-point rebuttal that was longer than Tyler’s original email, Sunny belittled everything from his grasp of statistics to his knowledge of laboratory science.

On the topic of proficiency testing, Sunny wrote:

“That reckless comment and accusation about the integrity of our company, its leadership and its core team members based on absolute ignorance is so insulting to me that had any other person made these statements, we would have held them accountable in the strongest way.  The only reason I have taken so much time away from work to address this personally is because you are Mr. Shultz’s grandson…

I have now spent an extraordinary amount of time postponing critical business matters to investigate your assertions—the only email on this topic I want to see from you going forward is an apology that I’ll pass on to other people including Daniel here.”

Tyler replied to Sunny with a one-sentence email saying he was resigning.  Before he even got to his car, Tyler’s mother called and blurted, “Stop whatever you’re about to do!”  Tyler explained that he had already resigned.

“That’s not what I mean.  I just got off the phone with your grandfather.  He said Elizabeth called him and told him that if you insist on carrying out your vendetta against her, you will lose.”

Tyler was dumbfounded.  Elizabeth was threatening him through his family, using his grandfather to deliver the message.

Tyler went to the Hoover Institution to meet with his grandfather. George listened to what Tyler had to say.  Finally, George told his grandson that he thought he was wrong in this case.

In the meantime, a patient order for a hepatitis C test had reached the lab and Erika refused to run it on the Edisons, writes Carreyrou.  The reagents for the hepatitis C test were expired.  Also, the Edisons hadn’t been recalibrated in awhile.  Erika and a coworker decided to use commercially available hepatitis kits called OraQuick HCV.  That had worked until the lab had run out of them.  They tried to order more, but Sunny had gotten upset and tried to block it.  Sunny also learned that it was Erika who had given Tyler the proficiency-testing results.  Sunny asked Erika to meet with him and then told her, “You need to tell me if you want to work here or not.”

Erika went to meet Tyler, who suggested that she join him for dinner at his grandfather’s house.  Perhaps having two people with similar experiences would be more persuasive.  Unfortunately, while Charlotte, George’s wife, seemed receptive and incredulous, George wasn’t buying it.

Tyler had noticed how much he doted on Elizabeth.  His relationship with her seemed closer than their own.  Tyler also knew that his grandfather was passionate about science.  Scientific progress would make the world a better place and save it from such perils as pandemics and climate change, he often told his grandson.  This passion seemed to make him unable to let go of the promise of Theranos.

George said a top surgeon in New York had told him the company was gong to revolutionize the field of surgery and this was someone his good friend Henry Kissinger considered to be the smartest man alive.  And according to Elizabeth, Theranos’s devices were already being used in medevac helicopters and hospital operating rooms, so they must be working.

Tyler and Erika tried to tell him that couldn’t possibly be true given that the devices were barely working within the walls of Theranos.  But it was clear they weren’t making any headway.  George urged them to put the company behind them and to move on with their lives. 

The next morning Erika quit Theranos.

 

FAME

After Theranos sued Richard Fuisz, Richard and Joe Fuisz resolved to fight it to the very end.  However, after they’d spent more than $2 million on their defense and after they realized how outgunned they were by Theranos’s lawyers—led by David Boies—they decided it would be better to settle.

It amounted to a complete capitulation on the Fuiszes’ part.  Elizabeth had won.

At a meeting with Boies, the two sides drafted the settlement agreement. Then Richard and Joe signed.

…Richard Fuisz looked utterly defeated.  The proud and pugnacious former CIA agent broke down and sobbed.

Roger Parloff, Fortune magazine’s legal correspondent, saw an article about the case involving Theranos and the Fuiszes.  Parloff called Dawn Schneider, Boies’s long-term public relations representative.  She offered to meet Parloff at his office.  On the walk across Midtown, Schneider thought that a better story to write about was Theranos and its brilliant young founder.  When she arrived at Parloff’s office, she told him about Theranos and said, “this is the greatest company you’ve never heard of.”

Parloff went to Palo Alto do meet with Elizabeth.

…what Elizabeth told Parloff she’d achieved seemed genuinely innovative and impressive.  As she and Sunny had stated to Partner fund, she told him the Theranos analyzer could perform as many as seventy different blood tests from one tiny finger-stick draw and she led him to believe that the more than two hundred tests on its menu were all finger-stick tests done with proprietary technology.  Since he didn’t have the expertise to vet her scientific claims, Parloff interviewed the prominent members of her board of directors and effectively relied on them as character witnesses… All of them vouched for Elizabeth emphatically. Shultz and Mattis were particularly effusive.

“Everywhere you look with this young lady, there’s a purity of motivation,” Shultz told him.  “I mean she is really trying to make the world better, and this is her way of doing it.”

Mattis went out his way to praise her integrity.  “She has probably one of the most mature and well-honed sense of ethics—personal ethics, managerial ethics, business ethics, medical ethics that I’ve ever heard articulated,” the retired general gushed.

Parloff’s cover story for Fortune magazine was published June 12, 2014.  Elizabeth instantly became a star.  Forbes then ran its own piece.

Two months later she graced one of the covers of the magazine’s annual Forbes 400 issue on the richest people in America.  More fawning stories followed in USA Today, Inc., Fast Company, and Glamour, along with segments on NPR, Fox Business, CNBC, CNN, and CBS News.  With the explosion of media coverage came invitations to numerous conferences and a cascade of accolades.  Elizabeth became the youngest person to win the Horatio Alger award.  Time magazine named her one of the one hundred most influential people in the world. President Obama appointed her a U.S. ambassador for global entrepreneurship, and Harvard Medical School invited her to join its prestigious board of fellows.

Carreyrou continues:

As much as she courted the attention, Elizabeth’s sudden fame wasn’t entirely her doing… In Elizabeth Holmes, the Valley had its first female billionaire tech founder.

Still, there was something unusual in the way Elizabeth embraced the limelight. She behaved more like a movie star than an entrepreneur, basking in the public adulation she was receiving.  Each week brought a new media interview or conference appearance.  Other well-known startup founders gave interviews and made public appearances too but with nowhere near the same frequency.  The image of the reclusive, ascetic young woman Parloff had been sold on had overnight given way to that of the ubiquitous celebrity.

Elizabeth excelled at delivering a heartwarming message that Theranos’s convenient blood tests could be used to catch diseases early so that no one would have to say goodbye to loved ones too soon, notes Carreyrou.  She soon started adding a new personal detail to her interviews and presentations: her uncle had died of cancer.

It was true that Elizabeth’s uncle, Ron Dietz, had died eighteen months earlier from skin cancer that had metastasized and spread to his brain.  But what she omitted to disclose was that she had never been close to him.  To family members who knew the reality of their relationship, using his death to promote her company felt phony and exploitative.

Of course, at that time, most people who heard Elizabeth in an interview or presentation didn’t know about the lies she was telling.  But she was a great salesperson.  Elizabeth told one story about a little girl who got stuck repeatedly because the nurse couldn’t find the vein.  Another story was about cancer patients depressed because of how much blood they had to give.

Patrick O’Neill, from TBWA/Chiat/Day, was Theranos’s chief creative officer.  He was raising Elizabeth’s profile and perfecting her image.

To Patrick, making Elizabeth the face of Theranos made perfect sense.  She was the company’s most powerful marketing tool.  Her story was intoxicating.  Everyone wanted to believe in it, including the numerous young girls who were sending her letters and emails.  It wasn’t a cynical calculus on his part: Patrick was one of her biggest believers.  He had no knowledge of the shenanigans in the lab and didn’t pretend to understand the science of blood testing.  As far as he was concerned, the fairy tale was real.

With over five hundred employees, Theranos had to move to a new location.  Patrick designed Elizabeth’s new office:

Elizabeth’s new corner office was designed to look like the Oval Office.  Patrick ordered a custom-made desk that was as deep as the president’s at its center but had rounded edges.  In front of it, he arranged two sofas and two armchairs around a table, replicating the White House layout.  At Elizabeth’s insistence, the office’s big windows were made of bulletproof glass.

 

THE HIPPOCRATIC OATH

Alan Beam had become disillusioned:

For his first few months as laboratory director, he’s clung to the belief that the company was going to transform with its technology.  But the past year’s events had shattered any illusion of that.  He now felt like a pawn in a dangerous game played with patients, investors, and regulators.  At one point, he’d had to talk Sunny and Elizabeth out of running HIV tests on diluted finger-stick samples.  Unreliable potassium and cholesterol results were bad enough.  False HIV results would have been disastrous.

Two of Alan’s colleagues had recently resigned out of disagreement with what they viewed as blatantly dishonest company policies.

One day Alan was talking with Curtis Schneider, one of the smartest people at Theranos, with a Ph.D. in inorganic chemistry and having spent four years as a postdoctoral scholar at Caltech. 

He told Curtis about the lab’s quality-control data and how it was being kept from him.  And he confided something else: the company was cheating on its proficiency testing.  In case Curtis hadn’t registered the implication of what he’d just said, he spelled it out: Theranos was breaking the law.

A few weeks later, Christian Holmes contacted Alan.

Christian wanted Alan to handle yet another doctor’s complaint.  Alan had fielded dozens of them since the company had gone live with its tests the previous fall.  Time and time again, he’d been asked to convince physicians that blood-test results he had no confidence in were sound and accurate.  He decided he couldn’t do it anymore.  His conscience wouldn’t allow him to.

He told Christian no and emailed Sunny and Elizabeth to inform them that he was resigning and to ask them to immediately take his name off the lab’s CLIA license.

December 15, 2014, there another article about Theranos in the New Yorker.  Adam Clapper, a pathologist in Columbia Missouri, who writes a blog about the industry Pathology Blawg, noticed the article.  He was very skeptical about Theranos.  Joe Fuisz noticed the article and told his father about it.  Richard read the article and got in touch with Adam.  Adam felt initially that he would need more proof.

A few days later, Richard noticed that someone named Alan Beam had looked at his LinkedIn profile.  Richard saw that Alan had been laboratory director at Theranos.  So he sent him an InMail, thinking it was worth a shot.  Alan got back to him.

Alan called and said to Richard, “You and I took the Hippocratic Oath, which is to first do no harm.  Theranos is putting people in harm’s way.”  Alan filled him in on all the details. 

Richard told Adam about what he’d learned from Alan.  Adam agreed that the information changed everything. However, he was worried about the legal liability of going against a $9 billion Silicon Valley company with a litigious history and represented by David Boies.  That said, Adam knew an investigative reporter at the Wall Street Journal.  John Carreyrou.

 

THE TIP

Adam called John Carreyrou at the Wall Street Journal.  Carreyrou says that even though nine times out of ten, tips don’t work out, he always listened because you never knew.  Also, he happened to have just finished a year-long investigation in Medicare fraud and he was looking for his next story.

February 26, 2015, Carreyrou reached Alan Beam.  Alan agreed to talk as long as his identity was kept confidential.

…the Theranos devices didn’t work.  They were called Edisons, he said, and they were error-prone.  They constantly failed quality-control.  Furthermore, Theranos used them for only a small number of tests.  It performed most of its tests on commercially available instruments and diluted the blood samples.

…Theranos didn’t want people to know its technology was limited, so it had contrived a way of running small finger-stick samples on conventional machines.  This involved diluting the finger-stick samples to make them bigger.  The problem, he said, was that when you diluted the samples, you lowered the concentration of analytes in the blood to a level the conventional machines could no longer measure accurately.

He said he had tried to delay the launch of Theranos’s blood tests in Walgreens stores and had warned Holmes that the lab’s sodium and potassium results were completely unreliable… I was barely getting my head around these revelations when Alan mentioned something called proficiency testing.  He was adamant that Theranos was breaking federal proficiency-testing rules.

There was more:

Alan also said that Holmes was evangelical about revolutionizing blood testing but that her knowledge base on science and medicine was poor, confirming my instincts.  He said she wasn’t the one running Theranos day-to-day.  A man named Sunny Balwani was.  Alan didn’t mince his words about Balwani: he was a dishonest bully who managed through intimidation.  Then he dropped another bombshell: Holmes and Balwani were romantically involved.

It’s not that there were rules against such a romantic involvement in the Silicon Valley startup world. Rather, it’s that Elizabeth was hiding the relationship from her board.  What other information might she be keeping from her board?

Alan told Carreyrou how he had brought up his concerns with Holmes and Balwani a number of times, but Balwani would either rebuff him or put him off, writes Carreyrou. 

Alan was most worried about potential harm to patients:

He described the two nightmare scenarios false blood-test results could lead to.  A false positive might cause a patient to have an unnecessary medical procedure.  But a false negative was worse: a patient with a serious condition that went undiagnosed could die.

Carreyrou experienced the familiar rush of a big reporting breakthrough, but he knew that he needed to get corroboration.  He proceeded to speak with others who had been associated with Theranos and who were willing to talk—some on the condition of anonymity.  A good start.  However, getting documentary evidence was “the gold standard for these types of stories.”  This would be harder.

Carreyrou spoke with Alan again.

Our conversation shifted to proficiency testing. Alan explained how Theranos was gaming it and he told me which commercial analyzers it used for the majority of its blood tests.  Both were made by Siemens… He revealed something else that hadn’t come up in our first call: Theranos’s lab was divided into two parts.  One contained the commercial analyzers and the other the Edison devices.  During her inspection of the lab, a state inspector had been shown only the part with the commercial analyzers.  Alan felt she’d been deceived.

He also mentioned that Theranos was working on a newer-generation device code-named 4S that was supposed to supplant the Edison and do a broader variety of blood tests, but it didn’t work at all and was never deployed in the lab.  Diluting finger-stick samples and running them on Siemens machines was supposed to be a temporary solution, but it had become a permanent one because the 4S had turned into a fiasco.

It was all beginning to make sense: Holmes and her company had overpromised and then cut corners when they couldn’t deliver. It was one thing to do that with software or a smartphone app, but doing it with a medical product that people relied on to make important health decisions was unconscionable.

Carreyrou reached out to twenty former and current Theranos employees.  Many didn’t respond.  Those Carreyrou got on the phone said they’d signed strict confidentiality agreements. They were worried about being sued.

Carreyrou’s initial conversations with Alan and two others had been “on deep background,” which meant Carreyrou could use what they said but had to keep their identities confidential.  Subsequently, he spoke with a former high-ranking employee “off the record.”  This meant that Carreyrou couldn’t make use of any information from that conversation. But Carreyrou did learn corroborating information even though it was off the record.  This further bolstered his confidence.

Carreyrou knew he needed proof that Theranos was delivering inaccurate blood-test results.  He discovered a doctor, Nicole Sundene, who had made a complaint about a Theranos blood test on Yelp.  Carreyrou met with Dr. Sundene, who told him about the experience of one of her patients, Maureen Glunz.

The lab report she’d received from Theranos had shown abnormally elevated results for calcium, protein, glucose, and three liver enzymes… Dr. Sundene had worried she might be on the cusp of a stroke and sent her straight to the hospital.  Glunz had spent four hours in the emergency room on the eve of Thanksgiving while doctors ran a battery of tests on her, including a CT scan.  She’d been discharged after a new set of blood tests performed by the hospital’s lab came back normal.  That hadn’t been the end of it, however.  As a precaution, she’d undergone two MRIs during the ensuing week…

When I met with Dr. Sundene at her office, I learned that Glunz wasn’t the only patient whose results she found suspect.  She told me more than a dozen of her patients had tested suspiciously high for potassium and calcium and she doubted the accuracy of those results as well.  She had written Theranos a letter to complain but the company hadn’t even acknowledged it.

Carreyrou met a Dr. Adrienne Stewart, who told him about two of her patients who’d gotten incorrect results from Theranos.  One patient had to delay a long-planned trip to Ireland because an incorrect result from Theranos suggested she could have deep vein thrombosis.  A second set of tests from another lab turned out to be normal.  Also, ultrasound of the patient’s legs didn’t reveal anything.

Another of Dr. Stewart’s patients had gotten a test result from Theranos indicating a high TSH value.

The patient was already on thyroid medication and the result suggested that he dose needed to be raised.  Before she did anything, Dr. Stewart sent the patient to get retested at Sonora Quest, a joint venture of Quest and the hospital system Banner Health.  The Sonora Quest result came back normal.  Had she trusted the Theranos result and increased the patient’s medication dosage, the outcome could have been disastrous, Dr. Stewart said.  The patient was pregnant.  Increasing her dosage would have made her levels of thyroid hormone too high and put her pregnancy at risk.

Carreyrou also met with Dr. Gary Betz.  He had a patient on medication to reduce blood pressure.  High potassium was one potential side effect of the medication, so Dr. Betz monitored it.  A Theranos test showed that his patient had an almost critical level of potassium.  A nurse sent Dr. Betz’s patient back to get retested.  But the phlebotomist was unable to complete the test despite three attempts to draw blood. Dr. Betz was very upset because if the initial test was accurate, an immediate change in the patient’s treatment was crucial.  He sent his patient to get tested as Sonora Quest.  The result came back normal.

As an experiment, Carreyrou and Dr. Sundene had each gotten their blood tested by Theranos and by another lab.  Carreyrou:

Theranos had flagged three of my values as abnormally high and one as abnormally low.  Yet on LabCorp’s report, all four of those values showed up as normal. Meanwhile, LabCorp had flagged both my total cholesterol and LDL cholesterol as high, while the Theranos described the first as “desirable” and the second as “near optimal.”

Those differences were mild compared to a whopper Dr. Sundene had found in her results.  According to Theranos, the amount of cortisol in her blood was less than one microgram per deciliter.  A value that low was usually associated with Addison’s disease, a dangerous condition characterized by extreme fatigue and low blood pressure that could result in death if it went untreated.  Her LabCorp report, however, showed a cortisol level of 18.8 micrograms per deciliter, which was within the normal range for healthy patients.  Dr. Sundene had no doubt which of the two values was the correct one.

Carreyrou mentions a “No surprise” rule they have at the Wall Street Journal.

We never went to press with a story without informing the story subject of every single piece of information we had gathered in our reporting and giving them ample time and opportunity to address and rebut everything.

Carreyrou met with Erica Cheung.

She said Theranos should never have gone live testing patient samples.  The company routinely ignored quality-control failures and test errors and showed a complete disregard for the well-being of patients, she said.  In the end, she had resigned because she was sickened by what she had become a party to, she told me.

Carreyrou also met with Tyler Shultz, who gave him a detailed account of his experiences with Theranos.  Finally, Carreyrou met with Rochelle Gibbons, the widow of Ian Gibbons.

I flew back to New York the next day confident that I’d reached a critical mass in my reporting and that it wouldn’t be too long before I could publish.  But that was underestimating whom I was up against.

 

THE AMBUSH

On May 27, 2015, Tyler went to his parents’ house for dinner, as he tried to do every two weeks.  His father, looking worried, asked Tyler if he’d spoken with an investigative journalist from the Wall Street Journal.  Yes, said Tyler.  His father: “Are you kidding me?  How stupid could you be?  Well, they know.”

His father told him that his grandfather George had called.  George said if Tyler wanted to get out of a “world of trouble,” he would have to meet with Theranos’s lawyers the next day and sign something.  Tyler called his grandfather and arranged to meet him later that night.

Carreyrou had sent a list of seven areas he wanted to discuss with Elizabeth to Matthew Traub, a representative of Theranos.  Included in one section was the coefficient of variation for one of the blood tests.  It happened to be a number that Tyler had calculated.  It was because of that number that Elizabeth had been able to tie Tyler to the investigative reporter.

However, the number Elizabeth tied to Tyler could have come from anyone.  When Tyler met with his grandfather, he categorically denied speaking with any reporter.  George told Tyler:  “We’re doing this for you.  Elizabeth says your career will be over if the article is published.”

Tyler summarized all the issues he had raised earlier regarding Theranos.  But his grandfather still didn’t agree with Tyler’s views.  George told his grandson that there was a one-page document Theranos wanted him to sign swearing confidentiality going forward.  Theranos argued that the Wall Street Journal article would include trade secrets of the company.  Tyler said he would consider signing the document if the company would stop bothering him. George then told Tyler that there were two Theranos lawyers upstairs.

Tyler felt betrayed because he had specifically asked to meet his grandfather with no lawyers.  His grandmother Charlotte told Tyler that she was questioning whether Theranos had a functioning product and that Henry Kissinger was also skeptical and wanted out.

The two lawyers, Mike Brill and Meredith Dearborn, were partners at Boies, Schiller & Flexner. Brille told Tyler he had identified him as a source for the Journal article.

He handed him three documents: a temporary restraining order, a notice to appear in court two days later, and a letter stating Theranos had reason to believe Tyler had violated his confidentiality obligations and was prepared to file suit against him.

Brille pressed Tyler to admit that he had spoken with a reporter.  Tyler kept denying it.  Brille kept pushing and pushing and pushing.  Finally, Tyler said the conversation needed to end.  His grandfather jumped in and defended Tyler and escorted the lawyers out of the house. 

[George] called Holmes and told her this was not what they had agreed upon.  She had sent over a prosecutor rather than someone who was willing to have a civilized conversation.  Tyler was ready to go to court the next day, he warned her.

George and Elizabeth reached a compromise.  George and Tyler would meet again at George’s house the following morning.  Tyler would look at the one-page document.  George asked Elizabeth to send a different lawyer.

The next morning, Tyler wasn’t surprised to see Brille again.  Brille had new documents. 

One of them was an affidavit stating that Tyler had never spoken to any third parties about Theranos and that he pledged to give the names of every current and former employee who he knew had talked to the Journal.  Brille asked Tyler to sign the affidavit.  Tyler refused.

George asked Tyler what it would take for him to sign it.  Tyler said Theranos would have to agree not to sue him.  George wrote the requirement on the affidavit.  Then he and Brille went into another room to talk.

In the interim, Tyler decided he wasn’t going to sign anything.  After speaking with two lawyers soon thereafter, Tyler stuck with his decision.  Brille had been threatening to sue immediately, but then told Tyler’s lawyer that they were going to delay the lawsuit in order to try to reach some agreement.

Tyler—through his lawyer—began exchanging drafts of the affidavit with Brille.  Tyler tried to make concessions in order to reach some agreement.  For instance, he agreed to be called a junior employee who couldn’t have known what he was talking about when it come to proficiency testing, assay validation, and lab operations.  But Theranos kept pushing Tyler to name the Journal’s other sources.  He refused.

As the stalemate dragged on, Boies Schiller resorted to the bare-knuckle tactics it had become notorious for.  Brille let it be known that if Tyler didn’t sign the affidavit and name the Journal‘s sources, the firm would make sure to bankrupt his entire family when it took him to court.  Tyler also received a tip that he was being surveilled by private investigators.

Tyler got a lawyer for his parents.  That way Tyler and his parents could communicate through attorneys and those conversations would be protected by attorney-client privilege.

This led to an incident that rattled both Tyler and his parents.  Hours after his parents’ new lawyer met with them for the first time, her car was broken into and her briefcase containing her notes from the meeting was stolen.

 

TRADE SECRETS

A Theranos delegation met Carreyrou at the offices of the Journal.  David Boies came with Mike Brille, Meredith Dearborn, and Heather King, who was now general counsel for Theranos.  Matthew Traub was there.  The only Theranos executive was Daniel Young.

Carreyrou brought along Mike Siconolfi, head of the Journal’s investigations team, and Jay Conti, the deputy general counsel of the Journal’s parent company.

Carreyrou had sent eighty questions, at Traub’s request, as a basis for the discussion.  King began the meeting by saying they were going to refute the “false premises” assumed by the questions.  The lawyers tried to intimidate Carreyrou.  King warned:

“We do not consent to your publication of our trade secrets.”

Carreyrou wasn’t going to be intimidated.  He retorted:

“We do not consent to waiving our journalistic privileges.”

King became more conciliatory as they agreed to start going through the questions one at a time.  Daniel Young was the only one there who could answer them.

After Young acknowledged that Theranos owned commercial blood analyzers, which he claimed the company used only for comparison purposes, rather than for delivering patient results, I asked if one of them was the Siemens ADVIA.  He declined to comment, citing trade secrets.  I then asked whether Theranos ran small finger-stick samples on the Siemens ADVIA with a special dilution protocol.  He again invoked trade secrets to avoid answering the question but argued that diluting blood samples was common in the lab industry.

Carreyrou pointed out that if they weren’t prepared to answer such basic questions that were at the heart of his story, what was the point of meeting?  Eventually Boies got angry and criticized Carreyrou’s reporting methods, saying he asked loaded questions to doctors.  Much more back-and-forth ensued between members of the Theranos delegation and Carreyrou, Siconolfi, and Conti.

How could anything involving a commercial analyzer manufactured by a third party possibly be deemed a Theranos trade secret?  I asked.  Brille replied unconvincingly that the distinction wasn’t as simple as I made it out to be.

Turning to the Edison, Carreyrou asked how many blood tests it performs.  The answer was that it was a trade secret.

I felt like I was watching a live performance of the Theater of the Absurd.

…It was frustrating but also a sign that I was on the right track. They wouldn’t be stonewalling if they had nothing to hide.

For four more hours, the meeting went on like this.  Young did answer a few questions.

He acknowledged problems with Theranos’s potassium test but claimed they had quickly been solved and none of the faulty results had been released to any patients.  Alan Beam had told me otherwise, so I suspected Young was lying about that. Young also confirmed that Theranos conducted proficiency testing differently than most laboratories but argued this was justified by the uniqueness of its technology. 

A few days later, Theranos threatened Erika Cheung with a lawsuit and also started started threatening Alan Beam again.  However, Alan had consulted a lawyer and felt less vulnerable to Theranos’s intimidation tactics.

Boies sent a twenty-three page letter to the Journal threatening a lawsuit if the paper published a story that defamed Theranos or revealed any of its trade secrets.  Boies attacked Carreyrou’s journalistic integrity.

His main evidence to back up that argument was signed statements Theranos had obtained from two of the other doctors I had spoken to claiming I had mischaracterized what they had told me and hadn’t made clear to them that I might use the information in a published article.  The doctors were Lauren Beardsley and Saman Rezaie…

The truth was that I hadn’t planned on using the patient case Dr. Beardsley and Rezaie had told me about because it was a secondhand account.  The patient in question was being treated by another doctor in their practice who had declined to speak to me.  But, while their signed statements in no way weakened my story, the likelihood that they had caved to the company’s pressure worried me.

Meanwhile, Dr. Stewart reassured Carreyrou that she was standing up for patients and for the integrity of lab testing.  She wouldn’t be pressured.  Balwani later told her that if the Journal article was published with Dr. Stewart in the story, her name would be dragged through the mud.  When Carreyrou spoke with Dr. Stewart, she asked him please not to use her name in the story.

 

LA MATTANZA

Roger Parloff of Fortune still believed in Theranos.  During an interview with Elizabeth for a second article he was working on, he asked about an Ebola test Theranos had been developing.

Given that an Ebola epidemic had been raging in West Africa for more than a year, Parloff thought a rapid finger-stick test to detect the deadly virus could be of great use to public health authorities and had been interested in writing about it.  Holmes said she expected to obtain emergency-use authorization for the test shortly and invited him to come see a live demonstration of it at Boies Schiller’s Manhattan offices.

Parloff arrived at the offices, and they told him they wanted to do two tests, one for Ebola and the other to measure potassium.  They pricked his finger twice.

Parloff wondered fleetingly why one of the devices couldn’t simultaneously perform both tests from a single blood sample but decided not to press the issue.

For some reason, the results of the tests were delayed.  An indicator of the machine’s progress seemed to be moving very slowly.

Balwani had tasked a Theranos software engineer named Michael Craig to write an application for the miniLab’s software that masked test malfunctions.  When something went wrong insider the machine, the app kicked in and prevented an error message from appearing on the digital display.  Instead, the screen showed the test’s progress slowing to a crawl.

[…]

In the absence of real validation data, Holmes used these demos to convince board members, prospective investors, and journalists that the miniLab was a finished working product.  Michael Craig’s app wasn’t the only subterfuge used to maintain the illusion. During demos at headquarters, employees would make a show of placing the finger-stick sample of a visiting VIP in the miniLab, wait until the visitor had left the room, and then take the sample out and bring it to a lab associate, who would run it on one of the modified commercial analyzers.

Parloff had no idea he’d been duped.

Back in California, Holmes had invited Vice President Joe Biden to visit the company’s facilities.

Holmes and Balwani wanted to impress the vice president with a vision of a cutting-edge, completely automated laboratory. So instead of showing him the actual lab, they created a fake one.

Carreyrou writes:

A few days later, on July 28, I opened that morning’s edition of the Journal and nearly spit out my coffee: as I was leafing through the paper’s first section, I stumbled across an op-ed written by Elizabeth Holmes crowing about Theranos’s herpes-test approval and calling for all lab tests to be reviewed by the FDA. She’d been denying me an interview for months, her lawyers had been stonewalling and threatening my sources, and here she was using my own newspaper’s opinion pages to perpetuate the myth that she was regulators’ best friend.

Of course, because of the firewall between the Journal’s news and editorial side, Paul Gigot and his staff had no idea what Carreyrou was working on.  Nonetheless, Carreyrou was annoyed because it seemed like Holmes was trying to make it more difficult for the paper to publish Carreyrou’s investigation.

Carreyrou went to speak with his editor, Mike Siconolfi, hoping they could speed up the publication of his Theranos article.  But Mike, who was Italian American, urged patience and then asked Carreyrou, “Did I ever tell you about la mattanza?”  La mattanza was an ancient Sicilian ritual in which fishermen waded into the Mediterranean Sea with clubs and spears.  Then they stood perfectly still for hours until the fish no longer noticed them.  Someone would give the signal and the fishermen would strike.

 

DAMAGE CONTROL

Soon after Carreyrou started investigating Theranos, the company completed another round of fund-raising.  They raised $430 million, $125 million of which came from Rupert Murdoch, who controlled News Corporation, the parent company of the Journal.

He was won over by Holmes’s charisma and vision but also by the financial projections she gave him.  The investment packet she sent forecast $330 million in profits on revenues of $1 billion in 2015 and $505 million in profits on revenues of $2 billion in 2016.  These numbers made what was now a $10 billion valuation seem cheap.

Murdoch also derived comfort from some of the other reputable investors he heard Theranos had lined up.  They included Cox Enterprises, the Atlanta-based, family-owned conglomerate whose chairman, Jim Kennedy, he was friendly with, and the Waltons of Walmart fame.  Other big-name investors he didn’t know about ranged from Bob Kraft, owner of the New England Patriots, to Mexican billionaire Carlos Slim and John Elkann, the Italian industrialist who controlled Fiat Chrystler Automobiles.

On two separate occasions when Holmes met with Murdoch, she brought up Carreyrou’s story, saying it was false and would damage Theranos.  Both times, Murdoch maintained that he trusted the Journal’s editors to handle the matter fairly.

Meanwhile, Theranos continued to try to intimidate Carreyrou’s sources.  For instance, two patients who had appointments with Dr. Sundene fabricated negative stories and posted them on Yelp.  Dr. Sundene had to hire an attorney to get Yelp to remove the bad reviews.

The Journal finally published Carreyrou’s story on the front page on Thursday, October 15, 2015.

The headline,“A Prized Startup Struggles,” was understated but the article itself was devastating.  In addition to revealing that Theranos ran all but a small fraction of its tests on conventional machines and laying bare its proficiency-testing shenanigans and its dilution of finger-stick samples, it raised serious questions about the accuracy of its own devices.  It ended with a quote from Maureen Glunz saying that “trial and error on people” was “not OK,” bringing home what I felt was the most important point: the medical danger to which the company had exposed patients.

The story sparked a firestorm…

Other news organization picked up the story and produced critical pieces.  In Silicon Valley, everyone was talking about the Theranos story.  Some, including venture capitalist Marc Andreesen, defended Theranos.  Others revealed that they had had their doubts for some time:

Why had Holmes always been so secretive about her technology?  Why had she never recruited a board member with even basic knowledge of blood science?  And why hadn’t a single venture capital firm with expertise in health care put money into the company?

Many others didn’t know what to believe.

Carreyrou writes:

We knew that the battle was far from over and that Theranos and Boies would be coming at us hard in the coming days and weeks.  Whether my reporting stood up to their attacks would largely depend on what actions, if any, regulators took.

Carreyrou was trying to speak with his source at the FDA and finally reached him:

On deep background, he confirmed to me that the FDA had recently conducted a surprise inspection of Theranos’s facilities in Newark and Palo Alto.  Dealing a severe blow to the company, the agency had declared its nanotainter an uncleared medical device and forbid it from continuing to use it, he said.

He explained that the FDA had targeted the little tube because, as a medical device, it clearly fell under its jurisdiction and gave it the most solid legal cover to take action against the company.  But the underlying reason for the inspection had been the poor clinical data Theranos had submitted to the agency in an effort to get it to approve its tests.  When the inspectors failed to find any better data on-site, the decision had been made to shut down the company’s finger-stick testing by taking away the nanotainter, he said.  That wasn’t all: he said the Centers for Medicare and Medicaid Services had also just launched its own inspection of Theranos.

Holmes tried to jump ahead of the story by stating that the nanotainter withdrawal was a voluntary decision.

We quickly published my follow-up piece online. Setting the record straight, it revealed that the FDA had forced the company to stop testing blood drawn from patients’ fingers and declared its nanotainter an “unapproved medical device.”  The story made the front page of the paper’s print edition the next morning, providing more fuel to what was now a full-blown scandal.

Holmes called a meeting of all company employees.

Striking a defiant tone, she told the assembled staff that the two articles the Journal had published were filled with falsehoods seeded by disgruntled former employees and competitors.  This sort of thing was bound to happen when you were working to disrupt a huge industry with powerful incumbents who wanted to see you fail, she said.  Calling the Journal a “tabloid,” she vowed to take the fight to the paper.

A senior hardware engineer asked Balwani to lead them in a chant.  A few months earlier, they’d done a certain chant directed at Quest and LabCorp.  Everyone remember this chant.

Balwani was glad to lead the chant again.  Several hundred employees chanted:

“Fuck you, Carrey-rou!  Fuck you, Carrey-rou!”

The following week, the Journal was hosting the WSJ D.Live conference at which Holmes was scheduled to be interviewed. 

Holmes came out swinging from the start.  That was no surprise: we had expected her to be combative.  What we hadn’t fully anticipated was her willingness to tell bald-faced lies in a public forum.  Not just once, but again and again during the half-hour interview.  In addition to continuing to insist that the nanotainter withdrawal had been voluntary, she said the Edison devices referred to in my stories were an old technology that Theranos hadn’t used in years.  She also denied that the company had ever used commercial lab equipment for finger-stick tests.  And she claimed that the way Theranos conducted proficiency-testing was not only perfectly legal, it has the express blessing of regulators.

The biggest lie, to my mind, was her categorical denial that Theranos diluted finger-stick samples before running them on commercial machines.

By this point, several prominent Silicon Valley figures were publicly criticizing the company.  John-Louis Gassee published a blog post in which he mentioned pointedly different blood-test results he received from Theranos and Stanford Hospital.  He wrote Holmes asking about the discrepancy, but never got a reply.

Shultz, Kissinger, Sam Nunn, and other ex-statesmen left the Theranos board and instead formed a board of counselors.  David Boies joined the Theranos board.

Within days, the Journal received a letter from Heather King demanding a retraction of the main points of the two articles, calling them “libelous assertions.”  David Boies stated that a defamation suit was likely.  The Journal received another letter demanding that it retain all documents concerning Theranos.

But if Theranos thought this saber rattling would make us stand down, it was mistaken.  Over the next three weeks, we published four more articles.  They revealed that Walgreens had halted a planned nationwide expansion of Theranos wellness centers, that Theranos had tried to sell more shares at a higher valuation days before my first story was published, that its lab was operating without a real director, and that Safeway had walked away from their previously undisclosed partnership over concerns about its testing.

In an interview with Bloomberg Businessweek, Holmes said she was the victim of sexism.

In the same story, her old Stanford professor, Channing Robertson, dismissed questions about he accuracy of Theranos’s testing as absurd, saying the company would have to be “certifiable” to go to market with a product that people’s lives depended on knowing that it was unreliable.  He also maintained that Holmes was a once-in-a-generation genius, comparing her to Newton, Einstein, Mozart, and Leonardo da Vinci.

Carreyrou comments:

There was only one way the charade would end and that was if CMS, the chief regulatory of clinical laboratories, took strong action against the company.  I needed to find out what had come of that second regulatory inspection.

 

THE EMPRESS HAS NO CLOTHES

Based on a complaint from Erika Cheung, a veteran CMS field inspector, Gary Yamamoto and his colleague Sarah Bennett made a surprise inspection of Theranos’s lab.  Yamamoto and Bennett planned to spend two days, but there were so many issues that they asked for more time.  Balwani asked if they could return in two months and they agreed.

In late 2015 and early 2016, Carreyrou tried to find out about the second inspection conducted by Yamamoto and Bennett.  Finally he learned that the CMS inspectors had found “serious deficiencies.”

How serious became clear a few days later when the agency released a letter it had sent the company saying they posed “immediate jeopardy to patient health and safety.”  The letter gave the company ten days to come up with a credible correction plan and warned that failing to come back into compliance quickly could cause the lab to lose its federal certification.

This was major.  The overseer of clinical laboratories in the United States had not only confirmed that there were significant problems with Theranos’s blood tests, it had deemed the problems grave enough to put patients in immediate danger.  Suddenly, Heather King’s written retraction demands, which had been arriving like clockwork after each story we published, stopped.

However, Theranos continued to minimize the seriousness of the situation.  In a statement, it claimed to have already addressed many of the deficiencies and that the inspection findings didn’t reflect the current state of the Newark lab.  It also claimed that the problems were confined to the way the lab was run and had no bearing on the soundness of its proprietary technology.  It was impossible to disprove these claims without access to the inspection report.  CMS usually made such documents public a few weeks after sending them to the offending laboratory, but Theranos was invoking trade secrets to demand that it be kept confidential…

Carreyrou filed a Freedom of Information Act request to try to force CMS to release the inspection report.

But Heather King continued to urge the agency not to make the report public without extensive redactions, claiming that doing so would expose valuable trade secrets.  It was the first time the owner of a laboratory under the threat of sanctions had demanded redactions to an inspection report, and CMS seemed unsure how to proceed.

Carreyrou finally got his hands on a copy of the CMS report.

For one thing, it proved that Holmes had lied at the Journal’s tech conference the previous fall: the proprietary devices Theranos used in the lab were indeed called “Edison,” and the report showed it had used them for only twelve of the 250 tests on its menu.  Every other test had been run on commercial analyzers.

More important, the inspection report showed, citing the lab’s own data, that the Edisons produced wildly erratic results.  During one month, they had failed quality-control checks nearly a third of the time.  One of the blood tests run on the Edisons, a test to measure a hormone that affects testosterone levels, had failed quality control an astounding 87 percent of the time.  Another test, to help detect prostrate cancer, had failed 22 percent of its quality-control checks.  In comparison runs using the same blood samples, the Edisons had produced results that differed from those of conventional machines by as much as 146 percent.  And just as Tyler Shultz had contended, the devices couldn’t reproduce their own results. An Edison test to measure vitamin B12 had a coefficient of variation that ranged from 34 to 48 percent, far exceeding the 2 or 3 percent common for the test at most labs.

As for the lab itself, it was a mess: the company had allowed unqualified personnel to handle patient samples, it had stored blood at the wrong temperatures, it had let reagents expire, and it had failed to inform patients of flawed test results, among many other lapses.

[…]

The coup de grace came a few days later when we obtained a new letter CMS had sent to Theranos.  It said the company had failed to correct forty-three of the forty-five deficiencies the inspectors had cited it for and threatened to ban Holmes from the blood-testing business for two years.

Carreyrou met up with Tyler Shultz.  Carreyrou points out that Tyler never buckled even though he was under enormous pressure.  Moreover, his parents spent over $400,000 on legal fees.  Were it not for Tyler’s courage, Carreyrou acknowledges that he might never have gotten his first Theranos article published.  In addition, Tyler continued to be estranged from his grandfather, who continued to believe Elizabeth and not Tyler.

Not long after this meeting between Tyler and Carreyrou, Theranos contacted Tyler’s lawyers and said they knew about the meeting.  Because neither Tyler nor Carreyrou had told anyone about the meeting, they realized they were under surveillance and being followed.  (Alan Beam and Erika Cheung were probably also under surveillance.)  At this juncture, Tyler wasn’t too worried, joking that next time he might take a selfie of himself and Carreyrou and sent it to Holmes “to save her the trouble of hiring PIs.”

Soon thereafter, there was more bad news for Theranos.  Carreyrou:

…we reported that Theranos had voided tens of thousands of blood-test results, including two years’ worth of Edison tests, in an effort to come back into compliance and avoid the CMS ban.  In other words, it had effectively admitted to the agency that not a single one of the blood tests run on its proprietary devices could be relied upon.  Once again, Holmes had hoped to keep the voided tests secret, but I found out about them from my new source, the one who had leaked to me CMS’s letter threatening to ban Holmes from the lab industry.  In Chicago, executives at Walgreens were astonished to learn of the scale of the test voidings.  The pharmacy chain had been trying to get answers from Theranos about the impact on its customers for months.  On June 12, 2016, it terminated the companies’partnership and shut down all the wellness centers located in its stores.

In another crippling blow, CMS followed through on its threat to ban Holmes and her company from the lab business in early July.  More ominously, Theranos was now the subject of a criminal investigation by the U.S. Attorney’s Office in San Francisco and of a parallel civil probe by the Securities and Exchange Commission. 

Many investors in Theranos were fed up:

Partner Fund, the San Francisco hedge fund that had invested close to $100 million in the company in early 2014, sued Holmes, Balwani, and the company in Delaware’s Court of Chancery, alleging that they had deceived it with “a series of lies, material misstatements, and omissions.” Another set of investors led by the retired banker Robert Coleman filed a separate lawsuit in federal court in San Francisco.  It also alleged securities fraud and sought class-action status.

Most of the other investors opted against litigation, settling instead for a grant of extra shares in exchange for a promise not to sue.  One notable exception was Rupert Murdoch.  The media mogul sold his stock back to Theranos for one dollar so he could claim a big tax write-off on his other earnings.  With a fortune estimated at $12 billion, Murdoch could afford to lose more than a $100 million on a bad investment.

[…]

Walgreens, which had sunk a total of $140 million into Theranos, filed its own lawsuit against the company, accusing it of failing to meet the “most basic quality standards and legal requirements” of the companies’ contract.  “The fundamental premise of the parties’contract—like any endeavor involving human health—was to help people, and not to harm them,” the drugstore chain wrote in its complaint.

Carreyrou concludes the chapter:

The number of test results Theranos voided or corrected in California and Arizona eventually reached nearly 1 million.  The harm done to patients from all those faulty tests is hard to determine.  Ten patients have filed lawsuits alleging consumer fraud and medical battery.  One of them alleges that Theranos’s blood tests failed to detect his heart disease, leading him to suffer a preventable heart attack.  The suits have been consolidated into a putative class action in federal court in Arizona.  Whether the plaintiffs are able to prove injury in court remains to be seen.

One thing is certain: the chances that people would have died from missed diagnoses or wrong medical treatments would have risen expontentially if the company had expanded its blood-testing services to Walgreen’s 8,134 other U.S. stores as it was on the cusp of doing when Pathology Blawg’s Adam Clapper reached out to me.

 

EPILOGUE

Theranos settled the Partners Fund case for $43 million, and it settled the Walgreens lawsuit for more than $25 million.  On March 14, 2018, the Securities and Exchange Commission charged Theranos, Holmes, and Balwani with conducting “an elaborate, years-long fraud.”

To resolve the agency’s civil charges, Holmes was forced to relinquish her voting control over the company, give back a big chunk of her stock, and pay a $500,000 penalty.  She also agreed to be barred from being an officer or director in a public company for ten years.  Unable to reach a settlement with Balwani, the SEC sued him in federal court in California. In the meantime, the criminal investigation continued to gather steam.  As of this writing, criminal indictments of both Holmes and Balwani on charges of lying to investors and federal officials seem a distinct possibility.

It’s one thing for a software or hardware company to overhype the arrival of its technology years before the product was ready.  The term “vaporware” describes this kind of software or hardware.  Microsoft, Apple,and Oracle were all accused of this at one point, observes Carreyrou.

But it’s crucial to bear in mind that Theranos wasn’t a tech company in the traditional sense.  It was first and foremost a health-care company.  Its product wasn’t software but a medical device that analyzed people’s blood.  As Holmes herself liked to point out in media interviews and public appearances at the height of her fame, doctors base 70 percent of their treatment decisions on lab results.  They rely on lab equipment to work as advertised.  Otherwise, patient health is jeopardized.

So how was Holmes able to rationalize gambling with people’s lives?

Carreyrou ends the book:

A sociopath is often described as someone with little or no conscience.  I’ll leave it to the psychologists to decide whether Holmes fits the clinical profile, but there’s no question that her moral compass was badly askew.  I’m fairly certain she didn’t initially set out to defraud investors and put patients in harm’s way when she dropped out of Stanford fifteen years ago.  By all accounts, she had a vision that she genuinely believed in and threw herself into realizing.  But in her all-consuming quest to be the second coming of Steve Jobs amid the gold rush of the “unicorn” boom, there came a point when she stopped listening to sound advice and began to cut corners.  Her ambition was voracious and it brooked no interference.  If there was collateral damage on her way to riches and fame, so be it.

 

BOOLE MICROCAP FUND

An equal weighted group of micro caps generally far outperforms an equal weighted (or cap-weighted) group of larger stocks over time.  See the historical chart here:  https://boolefund.com/best-performers-microcap-stocks/

This outperformance increases significantly by focusing on cheap micro caps.  Performance can be further boosted by isolating cheap microcap companies that show improving fundamentals.  We rank microcap stocks based on these and similar criteria.

There are roughly 10-20 positions in the portfolio.  The size of each position is determined by its rank.  Typically the largest position is 15-20% (at cost), while the average position is 8-10% (at cost).  Positions are held for 3 to 5 years unless a stock approaches intrinsic value sooner or an error has been discovered.

The mission of the Boole Fund is to outperform the S&P 500 Index by at least 5% per year (net of fees) over 5-year periods.  We also aim to outpace the Russell Microcap Index by at least 2% per year (net).  The Boole Fund has low fees.

If you are interested in finding out more, please e-mail me or leave a comment.

My e-mail: jb@boolefund.com

Disclosures: Past performance is not a guarantee or a reliable indicator of future results. All investments contain risk and may lose value. This material is distributed for informational purposes only. Forecasts, estimates, and certain information contained herein should not be considered as investment advice or a recommendation of any particular security, strategy or investment product. Information contained herein has been obtained from sources believed to be reliable, but not guaranteed. No part of this article may be reproduced in any form, or referred to in any other publication, without express written permission of Boole Capital, LLC.

The Signal and the Noise

October 23, 2022

I recently re-read Nate Silver’s book, The Signal and the Noise (The Penguin Press, 2012).

From a value investing point of view, it’s crucial to bear in mind that trying to forecast the stock market will typically cause you to make less money than you otherwise would.  It’s far more reliable and profitable over the long term to stay focused on individual businesses without ever trying to predict the market or the economy.

Yet it’s worth reviewing Silver’s book because it discusses Bayes’ rule, which is essential for anyone trying to make predictions.

Most of us, even many scientists, do a poor job when it comes to making predictions and when it comes to updating our beliefs.  To understand why we make these errors so often, it helps to recall that we have two different mental systems:

System 1:  Operates automatically and quickly;  makes instinctual decisions based on heuristics.

System 2:  Allocates attention (which has a limited budget) to the effortful mental activities that demand it, including logic, statistics, and complex computations.

Usually we rely on System 1 to make predictions.  Most of the time, these predictions are accurate because they deal with areas of life with very high predictability, like the fact that there is gravity.

But when we encounter complex phenomena where only the careful use of proper statistical thinking can help us make good decisions, System 1 nearly always makes mistakes.  In these situations, we have to slow down and consciously activate our System 2.

Once we’ve learned to activate System 2 when it is required, there are two separate steps we need to learn:

  • First, we must train ourselves to make good predictions on the basis of all available evidence.
  • Second, we must train ourselves to test our predictions and to update our hypotheses on the basis of new information.  This is where Bayes’ rule comes in.

Here is an outline for this blog:

  • Ignore Macro Forecasting; Focus on Individual Businesses
  • Scientific Progress
  • Out of Sample Events
  • Foxes vs. Hedgehogs
  • Thinking Very Big and Very Small
  • Chaos Theory
  • Earthquakes
  • Economic Forecasting
  • Bayes’ Rule
  • The Problem of False Positives
  • Conclusion

Note:  Bayes’ rule is in the running as the most important formula in artificial intelligence.  Although a market-beating AI value investor may be 10-20 years away, it’s interesting to follow some of the developments.

 

IGNORE MACRO FORECASTING;  FOCUS ON INDIVIDUAL BUSINESSES

No one has ever been able to predict the stock market with any sort of reliability.  Ben Graham, the father of value investing, had about a 200 IQ.  Buffett calls Graham “the smartest man I ever knew.”  Here is what Graham said about market forecasting:

… if I have noticed anything over these 60 years on Wall Street, it is that people do not succeed in forecasting what’s going to happen to the stock market.

If you’re a value investor buying individual businesses when their stocks are cheap, then macroeconomic variables generally aren’t relevant.  Furthermore, most investors and businesspeople that pay attention to political and economic forecasts end up worse off as a result.  Here are a few good quotes from Buffett on forecasting:

Market forecasters will fill your ear but never fill your wallet.

We will continue to ignore political and economic forecasts, which are an expensive distraction for many investors and businessmen.

Charlie and I never have an opinion on the market because it wouldn’t be any good and it might interfere with the opinions we have that are good.

If we find a company we like, the level of the market will not really impact our decisions.  We will decide company by company.  We spend essentially no time thinking about macroeconomic factors.  In other words, if somebody handed us a prediction by the most revered intellectual on the subject, with figures for unemployment or interest rates or whatever it might be for the next two years, we would not pay any attention to it.  We simply try to focus on businesses that we think we understand and where we like the price and management.

The great economist John Maynard Keynes developed a similar investment philosophy to that held by Buffett and Munger.  Though Keynes was a true genius, he failed twice trying to invest based on macro predictions.  Finally, he realized that a concentrated value investment approach was far more effective.

Keynes did very well over decades as a focused value investor.  His best advice:

  • Buy shares when they are cheap in relation to probable intrinsic value;
  • Ignore macro and market predictions, and stay focused on a few individual businesses that you understand and whose management you believe in;
  • Hold those businesses for many years as long as the investment theses are intact;
  • Try to have negatively correlated investments (for example, the stock of a gold miner, says Keynes).

 

SCIENTIFIC PROGRESS

One of Silver’s chief points in the book is that we have more data than ever before, but the signal is often overwhelmed by the noise.  Says Silver:

Data-driven predictions can succeed – and they can fail.  It is when we deny our role in the process that the odds of failure rise.  Before we demand more of our data, we need to demand more of ourselves. 

When it comes to demanding more of ourselves, what Philip Tetlock and Barbara Mellers are doing with The Good Judgment Project is very worthwhile:  http://www.goodjudgment.com/

Silver points out that if the underlying incidence of true hypotheses is low, then it’s quite likely we will have many false positives.  John Ioannidis has already shown this – with respect to medical research – in his 2005 paper, ‘Why Most Published Research Findings Are False.’  Silver spoke with Ioannidis, who said:

I’m not saying that we haven’t made any progress.  Taking into account that there are a couple of million papers, it would be a shame if there wasn’t.  But there are obviously not a couple of million discoveries.  Most are not really contributing much to generating knowledge.

 

OUT OF SAMPLE EVENTS

There were many failures of prediction related to the 2008 financial crisis.  Silver observes that there is a common thread to these failures:

  • The confidence that homeowners had about housing prices may have stemmed from the fact that there had not been a substantial decline in U.S. housing prices in the recent past. However, there had never before been such a widespread increase in U.S. housing prices like the one that preceded the collapse.
  • The confidence that the banks had in Moody’s and S&P’s ability to rate mortgage-backed securities may have been based on the fact that the agencies had generally performed competently in rating other types of financial assets. However, the ratings agencies had never before rated securities as novel and complex as credit default options.
  • The confidence that economists had in the ability of the financial system to withstand a housing crisis may have arisen because housing price fluctuations had generally not had large effects on the financial system in the past. However, the financial system had probably never been so highly leveraged, and it had certainly never made so many side bets on housing before.
  • The confidence that policy makers had in the ability of the economy to recuperate quickly from the financial crisis may have come from their experience of recent recessions, most of which had been associated with rapid, ‘V-shaped’ recoveries. However, those recessions had not been associated with financial crisis, and financial crises are different.

Silver explains that these events were out of sample, which was a major reason for the failed forecasts.  The problem is that few forecasters ever want to look for examples and evidence outside of what their models have already considered:

We will be forced to acknowledge that we know less about the world than we thought we did.  Our personal and professional incentives almost always discourage us from doing this.

We forget – or we willfully ignore – that our models are simplifications of the world.  We figure that if we make a mistake, it will be at the margin.

In complex systems, however, mistakes are not measured in degrees but in whole orders of magnitude…

One of the pervasive risks that we face in the information age… is that even if the amount of knowledge in the world is increasing, the gap between what we know and what we think we know may be widening.  This syndrome is often associated with very precise-seeming predictions that are not at all accurate.

 

FOXES VS. HEDGEHOGS

Silver tabulates Philip Tetlock’s descriptions of foxes versus hedgehogs. (Tetlock is the author of Expert Political Judgment: How Good Is It? How Can We Know? and also Superforecasting: The Art and Science of Prediction.)

How Hedgehogs Think

  • Specialized: Often have spent the bulk of their careers on one or two great problems.  May view the opinions of ‘outsiders’ skeptically.
  • Stalwart: Stick to the same ‘all-in’ approach – new data is used to refine the original model.
  • Stubborn: Mistakes are blamed on bad luck or idiosyncratic circumstances – a good model had a bad day.
  • Order-seeking: Expect that the world will be found to abide by relatively simple governing relationships once the signal is identified through the noise.
  • Confident: Rarely hedge their predictions and are reluctant to change them.
  • Ideological: Expect that solutions to many day-to-day problems are manifestations of some grander theory or struggle.

How Foxes Think

  • Multidisciplinary: Incorporate ideas from different disciplines and regardless of their origin on the political spectrum.
  • Adaptable: Find a new approach – or pursue multiple approaches at the same time – if they aren’t sure the original one is working.
  • Self-critical: Sometimes willing (if rarely happy) to acknowledge mistakes in their predictions and accept the blame for them.
  • Tolerant of complexity: See the universe as complicated, perhaps to the point of many fundamental problems being irresolvable or inherently unpredictable.
  • Cautious: Express their predictions in probabilistic terms and qualify their opinions.
  • Empirical: Rely more on observation than theory.

Foxes are better forecasters than hedgehogs.  But hedgehogs – because of their big, bold predictions – are much more likely to be interviewed on television.

Silver describes three broad principles that he relies on in the FiveThirtyEight forecasting model:

Principle 1:  Thinking Probabilistically

Each forecast comes with a range of possible outcomes.  The distribution of possible outcomes is an honest expression of the uncertainty that exists in the real world.  What typifies a good forecaster is that the range of possible outcomes is itself supported by the later results of the forecasts.  In other words, if you examine all the times when a good forecaster said there was a 90 percent chance of an event happening, those predicted events should have happened about 90 percent of the time.

Foxes very often give a range of possible outcomes, while hedgehogs rarely do.

Principle 2:  Update Your Forecasts

When good forecasters get new information that changes the probabilities associated with their prediction, they update their prediction accordingly.  A fox has no trouble changing her mind if that’s what the new evidence suggests.

Unfortunately, some people think changing one’s mind on the basis of new evidence is a sign of weakness.  But if the forecaster is simply incorporating new information as well as possible, that’s a sign of strength, not weakness.  Silver quotes John Maynard Keynes:

When the facts change, I change my mind.  What do you do, sir?

Principle 3:  Look for Consensus

Very often the consensus estimate is better than most (and sometimes all) individual forecasts:

Quite a lot of evidence suggests that aggregate or group forecasts are more accurate than individual ones, often somewhere between 15 and 20 percent more accurate depending on the discipline.

A common experiment is to present a group of at least thirty people with a jar of pennies, and then ask each person in the group to guess how many pennies are in the jar.  In nearly every case, the average guess of the group is more accurate than every individual guess.

Stock prices can be thought of in this way.  But there are exceptions occasionally.

The lesson for the fox – in addition to recognizing when the aggregate is likely the best estimate – is to attempt to implement a process of aggregation within your own mind.  Try to incorporate as many different types of information and points of view as possible in the process of developing a prediction.

 

THINKING VERY BIG AND VERY SMALL

Sometimes innovation is very incremental, while other times it involves a big jump forward:

Good innovators think very big and they think very small.  New ideas are sometimes found in the most granular details of a problem where few others bother to look.  And they are sometimes found when you are doing your most abstract and philosophical thinking, considering why the world is the way that it is and whether there might be an alternative to the dominant paradigm.  Rarely can they be found in the temperate latitudes between these two spaces, where we spend 99 percent of our lives.  The categorizations and approximations we make in the normal course of our lives are usually good enough to get by, but sometimes we let information that might give us a competitive advantage slip through the cracks.

Most great forecasters constantly innovate and improve.

 

CHAOS THEORY

Silver explains how chaos theory applies to systems in which two properties hold:

  • The systems are dynamic, meaning that the behavior of the system at one point in time influences its behavior in the future;
  • And they are nonlinear, meaning they abide by exponential rather than additive relationships.

Trying to predict the weather is trying to predict a chaotic system:

The problem begins when there are inaccuracies in our data… Imagine that we’re supposed to be taking the sum of 5 and 5, but we keyed in the second number wrong.  Instead of adding 5 and 5, we add 5 and 6.  That will give us an answer of 11 when what we really want is 10.  We’ll be wrong, but not by much: addition, as a linear operation, is pretty forgiving.  Exponential operations, however, extract a lot more punishment when there are inaccuracies in our data.  If instead of taking 5 to the 5th power – which should be 3,215 – we instead take 5 to the 6th power, we wind up with an answer of 15,625.  That’s way off: we’ve missed our target by 500 percent.

This inaccuracy quickly gets worse if the process is dynamic, meaning that our outputs at one stage of the process become our inputs in the next.  For instance, say that we’re supposed to take five to the fifth, and then take whatever result we get and apply it to the fifth power again.  If we’d made the error described above, and substituted a 6 for the second 5, our results will now be off by a factor of more than 3,000.  Our small, seemingly trivial mistake keeps getting larger and larger.

The weather is the epitome of a dynamic system, and the equations that govern the movement of atmospheric gases and fluids are nonlinear – mostly differential equations.  Chaos theory therefore most definitely applies to weather forecasting, making the forecasts highly vulnerable to inaccuracies in our data.

Sometimes these inaccuracies arise as the result of human error.  The more fundamental issue is that we can only observe our surroundings with a certain degree of precision.  No thermometer is perfect, and if it’s off in even the third or the fourth decimal place, this can have a profound impact on the forecast.

Silver notes that perhaps the most impressive improvements have been in hurricane forecasting.  Twenty-five years ago, the National Hurricane Center missed by an average of 350 miles when it forecasted a hurricane’s landfall three days in advance.  Today the average miss is only about one hundred miles.  (Forecasters have not gotten much better at forecasting hurricane intensity, however, since the forces that govern intensity occur at a much smaller scale.)

 

EARTHQUAKES

Seismologists have specific definitions for prediction and forecast:

  • A prediction is a definitive and specific statement about when and where an earthquake will strike: a major earthquake will hit Kyoto, Japan, on June 28.
  • Whereas a forecast is a probabilistic statement, usually over a longer time scale: there is a 60 percent chance of an earthquake in Southern California over the next thirty years.  (149)

The United States Geological Survey (USGS) holds that earthquakes cannot be predicted, but they can be forecasted.  Silver includes the following table in his book:

FIGURE 5-2.  FREQUENCY OF A MAJOR (>= MAGNITUDE 6.75) EARTHQUAKE WITHIN A 50-MILE RADIUS

Anchorage 1 per 30 years
San Francisco 1 per 35 years
Los Angeles 1 per 40 years
Seattle 1 per 150 years
Sacramento 1 per 180 years
San Diego 1 per 190 years
Salt Lake City 1 per 200 years
Portland, OR 1 per 500 years
Charleston, SC 1 per 600 years
Las Vegas 1 per 1,200 years
Memphis 1 per 2,500 years
Phoenix 1 per 7,500 years
New York 1 per 12,000 years
Boston 1 per 15,000 years
Philadelphia 1 per 17,000 years
St. Louis 1 per 23,000 years
Atlanta 1 per 30,000 years
Denver 1 per 40,000 years
Washington, DC 1 per 55,000 years
Chicago 1 per 75,000 years
Houston 1 per 100,000 years
Dallas 1 per 130,000 years
Miami 1 per 140,000 years

 

According to the Gutenberg-Richter law, for every increase of one point in magnitude, an earthquake is ten times less frequent.  Thus, given information on past earthquakes and their magnitudes in a given area, it’s straightforward to predict the frequency of more powerful earthquakes in the same area.

As far as specific predictions are concerned, however, weather forecasters are much further along than seismologists.  Weather forecasters have been able to develop a good theoretical understanding of the earth’s atmosphere because they can observe a great deal of it.  Seismologists, on the other hand, are trying to predict the results of events that mostly occur fifteen kilometers below the earth’s surface.  So it’s far more difficult for seismologists to develop a model of what is actually happening.

Overfitting: The Most Important Scientific Problem You’ve Never Heard Of

Mistaking noise for a signal is overfitting.  If the model fits past observations too loosely, it is underfitting.  If the model fits past observations too closely, it is overfitting.  Overfitting is a much more common error than underfitting, as Silver describes:

This seems like an easy mistake to avoid, and it would be if only we were omniscient and always knew about the underlying structure of the data.  In almost all real-world applications, however, we have to work by induction, inferring the structure from the available evidence.  You are most likely to overfit a model when the data is limited and noisy and when your understanding of the fundamental relationships is poor;  both circumstances apply in earthquake forecasting.

…Overfitting represents a double whammy: it makes our model look better on paper but perform worse in the real world.  Because of the latter trait, an overfit model eventually will get its comeuppance if and when it is used to make real predictions.  Because of the former, it may look superficially more impressive until then, claiming to make very accurate and newsworthy predictions and to represent an advance over previously applied techniques.  This may make it easier to get the model published in an academic journal or to sell to a client, crowding out more honest models from the marketplace.  But if the model is fitting noise, it has the potential to hurt science.

… To be clear, these mistakes are usually honest ones.  To borrow the title of another book, they play into our tendency to be fooled by randomness.

 

ECONOMIC FORECASTING

Economists have a poor track record of predicting recessions.  But many of them may not have good incentives to improve.

Silver examined, from 1993 to 2010, economic forecasts of GDP as stated by economists in the Survey of Professional Forecasters.  The Survey is unique in that it asks economists to give a range of outcomes and associated probabilities.  If economists’ forecasts were as accurate as they thought, then from 1993 to 2010, only 2 forecasts out of 18 would fall outside their prediction intervals.  But in fact, actual GDP fell outside the prediction intervals 6 times out of 18.

If you examine how economic forecasts have actually performed, writes Silver, then a 90 percent prediction interval spans about 6.4 points of GDP:

When you hear on the news that GDP will grow by 2.5 percent next year, that means it could quite easily grow at a spectacular rate of 5.7 percent instead.  Or it could fall by 0.7 percent – a fairly serious recession.  Economists haven’t been able to do any better than that, and there isn’t much evidence that their forecasts are improving.

Silver met with the economist Jan Hatzius, who has been somewhat more accurate in his forecasts (in 2007, he warned about the 2008 crisis).  Silver quotes Hatzius:

Nobody has a clue.  It’s hugely difficult to forecast the business cycle.  Understanding an organism as complex as the economy is very hard.

Silver lists three fundamental challenges economists face, according to Hatzius:

  • First, it is very hard to determine cause and effect from economic statistics alone.
  • Second, the economy is always changing, so explanations of economic behavior that hold in one business cycle may not apply to future ones.
  • Third, as bad as their forecasts have been, the data that economists have to work with isn’t much good either.

Some data providers track four million statistics on the U.S. economy.  But there have only been eleven recessions since the end of World War II.  Silver:

If you have a statistical model that seeks to explain eleven outputs but has to choose from among four million inputs to do so, many of the relationships it identifies are going to be spurious.  (This is another classic case of overfitting – mistaking noise for a signal…)

For example, the winner of the Super Bowl correctly ‘predicted’ the direction of the stock market in 28 out of 31 years (from 1967 thru 1997).  A test of statistical significance would have said that there was only a 1 in 4,700,000 possibility that the relationship was due to chance alone, says Silver.

…of the millions of statistical indicators in the world, a few will have happened to correlate especially well with stock prices or GDP or the unemployment rate.  If not the winner of the Super Bowl, it might be chicken production in Uganda.  But the relationship is merely coincidental.

Economic variables that are leading indicators in one economic cycle are often lagging indicators in the next economic cycle.

An Economic Uncertainty Principle

Feedback loops between economic forecasts and economic policy can be particularly problematic for economic forecasters.  If the economy looks like it’s at risk of going into recession, then the government and the Federal Reserve will take steps to lessen that risk, perhaps even averting a recession that otherwise would have occurred.

Not only do you have to forecast both the economy and policy responses.  But even when you examine past economic data, you have to take into account government policy decisions in place at the time, notes Silver.  This issue was first highlighted by economist Robert Lucas in 1976.  Silver continues:

Thus, it may not be enough to know what current policy makers will do;  you also need to know what fiscal and monetary policy looked like during the Nixon administration.  A related doctrine known as Goodhart’s law, after the London School of Economics professor who proposed it, holds that once policy makers begin to target a particular variable, it may begin to lose its value as an economic indicator….

At its logical extreme, this is a bit like the observer effect (often mistaken for a related concept, the Heisenberg uncertainty principle):  once we begin to measure something, its behavior starts to change.  Most statistical models are built on the notion that there are independent variables and dependent variables, inputs and outputs, and they can be kept pretty much separate from one another.  When it comes to the economy, they are all lumped together in one hot mess.

An Ever-Changing Economy

An even more fundamental problem is that the American and global economies are always evolving.  Even if you correctly grasp the relationships between different economic variables in the past, those relationships can change over the course of time.

Perhaps you correctly account for the fact that the U.S. economy now is dominated more by the service sector.  But how do you account for the fact that major central banks have printed trillions of dollars?  How do you account for interest rates near zero (or even negative)?

The U.S. stock market seems high based on history.  But if rates stay relatively low for the next 5-10, the U.S. stock market could gradually move higher from here.  U.S. stocks may even turn out, in retrospect, to be cheap today if there has been a structural shift to lower interest rates.

Furthermore, as Silver points out, you never know the next paradigm shift that will occur.  Will the future economy, or the future stock market, be less volatile or more?  What if breakthroughs in technology create a much wealthier economy where the need for many forms of human labor is significantly curtailed?  Is that the most likely way that debt levels can be reduced and interest rates can move higher?  Or will central banks inflate away most of the current debt by printing even more money and/or by keeping rates very low for many more years?  No one really knows.

Economic Data is Very Noisy

Most economic data series are subject to revision.  Average GDP could be revised up to very high GDP or revised down to a recession.  Silver:

So we should have some sympathy for economic forecasters.  It’s hard enough to know where the economy is going.  But it’s much, much harder if you don’t know where it is to begin with.

 

BAYES’ RULE

Eliezer Yudkowsky of the Machine Intelligence Research Institute provides an excellent intuitive explanation of Bayes’s rule:  http://www.yudkowsky.net/rational/bayes

Yudkowsky begins by discussing a situation that doctors often encounter:

1% of women at age forty who participate in routine screening have breast cancer.  80% of women with breast cancer will get positive mammographies.  9.6% of women without breast cancer will also get positive mammographies.  A woman in this age group had a positive mammography in a routine screening.  What is the probability that she actually has breast cancer?

Most doctors estimate the probability between 70% and 80%, which is wildly incorrect.

In order to arrive at the correct answer, Yudkowsky asks us to think of the question as follows.  We know that 1% of women at age forty who participate in routine screening have breast cancer.  So consider 10,000 women who participate in routine screening:

  • Group 1: 100 women with breast cancer.
  • Group 2: 9,900 women without breast cancer.

After the mammography, the women can be divided into four groups:

  • Group A: 80 women with breast cancer, and a positive mammography.
  • Group B: 20 women with breast cancer, and a negative mammography.
  • Group C: 950 women without breast cancer, and a positive mammography.
  • Group D: 8,950 women without breast cancer, and a negative mammography.

So the question again:  If a woman out of this group of 10,000 women has a positive mammography, what is the probability that she actually has breast cancer?

The total number of women who had positive mammographies is 80 + 950 = 1,030.  Of that total, 80 women had positive mammographies AND have breast cancer.  In looking at the total number of positive mammographies (1,030), we know that 80 of them actually have breast cancer.

So if a woman out of the 10,000 has a positive mammography, the chances that she actually has breast cancer = 80/1030  or 0.07767 or 7.8%.

That’s the intuitive explanation.  Now let’s look at Bayes’ Rule:

P(A|B) = [P(B|A) P(A)] / P(B)

Let’s apply Bayes’ Rule to the same question above:

1% of women at age forty who participate in routine screening have breast cancer.  80% of women with breast cancer will get positive mammographies.  9.6% of women without breast cancer will also get positive mammographies.  A woman in this age group had a positive mammography in a routine screening.  What is the probability that she actually has breast cancer?

P(A|B) = the probability that the woman has breast cancer (A), given a positive mammography (B)

Here is what we know:

P(B|A) = 80% – the probability of a positive mammography (B), given that the woman has breast cancer (A)

P(A) = 1% – the probability that a woman out of the 10,000 screened actually has breast cancer

P(B) = (80+950) / 10,000 = 10.3% – the probability that a woman out of the 10,000 screened has a positive mammography

Bayes’ Rule again:

P(A|B) = [P(B|A) P(A)] / P(B)

P(A|B) = [0.80*0.01] / 0.103 = 0.008 / 0.103 = 0.07767 or 7.8%

Derivation of Bayes’ Rule:

Bayesians consider conditional probabilities as more basic than joint probabilities.  You can define P(A|B) without reference to the joint probability P(A,B).  To see this, first start with the conditional probability formula:

P(A|B) P(B) = P(A,B)

but by symmetry you get:

P(B|A) P(A) = P(A,B)

It follows that:

P(A|B) = [P(B|A) P(A)] / P(B)

which is Bayes’ Rule.

 

THE PROBLEM OF FALSE POSITIVES

In the case of the age forty women who had a positive mammogram, we saw that only about 7.8% actually had cancer.  So there were many false positives.  Out of 10,000 age forty women tested, 950 tested positive but did not have cancer.

Silver explains how, in the Era of Big Data, if you look at published scientific results, there are likely to be many false positives.  Assume that 100 out of 1,000 hypotheses are actually true.  Further assume that 80% of true scientific hypotheses are correctly deemed to be true, while 90% of false hypotheses are correctly rejected.  So now we have four groups:

  • True positives: 80 of 100 hypotheses that are true are correctly deemed true
  • False negatives: 20 of 100 hypotheses that are true are incorrectly deemed false
  • False positives: 90 of 900 hypotheses that are false are incorrectly deemed true
  • True negatives: 810 of 900 hypotheses that are false are correctly deemed false

So you can see, under these assumptions, that we’ll have 80 true hypotheses correctly identified as true, but 90 false hypotheses incorrectly identified as true.  Silver comments:

…as we know from Bayes’ theorem, when the underlying incidence of something in a population is low (breast cancer in young women; truth in a sea of data), false positives can dominate the results if we are not careful.

 

CONCLUSION

Most of us, including scientists, are not very good at making probability estimates about future events.  But there are two pieces of good news, writes Silver:

  • First, our estimates are just a starting point. Bayes’ theorem will allow us to improve our estimates every time we get new information.
  • Second, with practice – and trial and error – we can get much better at making probability estimates in the first place. For instance, see:  http://www.goodjudgment.com/

Silver explains the importance of testing our ideas:

Bayes’ theorem encourages us to be disciplined about how we weigh new information.  If our ideas are worthwhile, we ought to be willing to test them by establishing falsifiable hypotheses and subjecting them to a prediction.  Most of the time, we do not appreciate how noisy the data is, and so our bias is to place too much weight on the newest data point…

But we can have the opposite bias when we become too personally or professionally invested in a problem, failing to change our minds when the facts do.  If an expert is one of Tetlock’s hedgehogs, he may be too proud to change his forecast when the data is incongruous with his theory of the world.

The more often you are willing to test your ideas, the sooner you can begin to avoid these problems and learn from your mistakes… It’s more often with small, incremental, and sometimes even accidental steps that we make progress.

 

BOOLE MICROCAP FUND

An equal weighted group of micro caps generally far outperforms an equal weighted (or cap-weighted) group of larger stocks over time.  See the historical chart here:  https://boolefund.com/best-performers-microcap-stocks/

This outperformance increases significantly by focusing on cheap micro caps.  Performance can be further boosted by isolating cheap microcap companies that show improving fundamentals.  We rank microcap stocks based on these and similar criteria.

There are roughly 10-20 positions in the portfolio.  The size of each position is determined by its rank.  Typically the largest position is 15-20% (at cost), while the average position is 8-10% (at cost).  Positions are held for 3 to 5 years unless a stock approaches intrinsic value sooner or an error has been discovered.

The mission of the Boole Fund is to outperform the S&P 500 Index by at least 5% per year (net of fees) over 5-year periods.  We also aim to outpace the Russell Microcap Index by at least 2% per year (net).  The Boole Fund has low fees.

 

If you are interested in finding out more, please e-mail me or leave a comment.

My e-mail: jb@boolefund.com

 

 

 

Disclosures: Past performance is not a guarantee or a reliable indicator of future results. All investments contain risk and may lose value. This material is distributed for informational purposes only. Forecasts, estimates, and certain information contained herein should not be considered as investment advice or a recommendation of any particular security, strategy or investment product. Information contained herein has been obtained from sources believed to be reliable, but not guaranteed. No part of this article may be reproduced in any form, or referred to in any other publication, without express written permission of Boole Capital, LLC.

The Ugliest Stocks Are the Best Stocks

October 16, 2022

Quantitative value investor Tobias Carlisle has written an excellent book entitled Deep Value: Why Activists and Other Contrarians Battle for Control of LOSING Corporations (Wiley, 2014).

The book has many important and counterintuitive lessons for quantitative value investors.  This blog post is a brief summary of some chief lessons.

 

UGLY VALUE STOCKS OUTPERFORM

Companies with low growth or no growth that are trading at cheap valuations significantly outperform companies with high growth.  In other words, as a group, companies that have been doing terribly and that are trading at cheap prices – seemingly justifiably – do much better than companies that have been doing well and growing fast.

It’s important to note that these findings apply to groups of stocks, not individual stocks.  Cheap value stocks, as a group (and as a portfolio), significantly outperform companies that have been doing well and whose stocks have been doing well.  Moreover, in states of the world including bear markets and recessions, value stocks do better than growth stocks.  So value stocks are less risky as a group than growth stocks.

On an individual level, a value stock is riskier than an average stock.  Whereas an average stock has a 50% chance of underperforming the market, a value stock – if it is just chosen based on cheapness alone, without additional criteria – has a greater than 50% chance of underperforming the market.  But as a group (and as a portfolio), value stocks – when compared to either growth stocks or average stocks – are less risky and perform better over time.

If you are following a deep value approach, there are additional criteria that you can apply in the stock selection process to reduce the percentage of deep value stocks that underperform the market.  One example is the Piotroski F-Score, which identifies companies that show improving fundamentals (e.g., increased cash flows or reduced debt levels).  Joseph Piotroski came up with the F-Score because he discovered that while value stocks outperform as a group, there are many individual value stocks dragging down the overall performance of the value portfolio.

In sum, a portfolio of ugly value stocks far outperforms the market over time.  Remarkably, this already significant outperformance can be noticeably improved by using the Piotroski F-Score to cut off the left tail of the return distribution.  See: https://boolefund.com/joseph-piotroski-value-investing/

 

STAGNANT VALUE OUTPERFORMS GROWING VALUE

If you only look at value stocks, which as a group outperform, doesn’t it make sense to focus on the cheap stocks where the companies have been doing well – in terms of growth – rather than the cheap stocks where the companies have been doing terribly?  No.  Carlisle comments:

This is a fascinating finding.  Intuitively, we are attracted to high growth and would assume that high-growth value stocks are high-quality stocks available at a bargain price.  The data show, however, that the low- or no-growth value stocks are the better bet.  It seems that the uglier the stock, the better the return, even when the valuations are comparable.

This same logic also applies to excellent, A+ companies versus unexcellent, D companies.  Carlisle again:

Buying well-run companies with good businesses seems to make so much sense.  Buying well-run companies with good businesses at bargain prices seems to make even more sense.  The research shows, however, that the better investment – rather than the better company – is the value stock, the scorned, the unexcellent, the Ds, the loss-making net nets.  And the better value stock, according to Lakonishok, Shleifer, and Vishny’s research, is the low- or ­no-growth value stock, what they describe as ‘contrarian value,’ and what I regard as deep value; the ugliest of the ugly.

Link to the famous 1994 paper by Josef Lakonishok, Andrei Schleifer, and Robert Vishny: http://scholar.harvard.edu/files/shleifer/files/contrarianinvestment.pdf?m=1360042367

 

QUANTITATIVE DEEP VALUE

Investing in general is difficult to do effectively, which is why Warren Buffett advises most people to invest in low-cost index funds.

Deep value investing can be even more difficult because it requires consistently buying what everyone else hates – the ugliest, the worst, the cheapest stocks available.  Deep value stocks are almost always facing enormous business problems, and quite often the industries in which deep value stocks are found are doing horribly.

Thus, the best way for most investors to benefit from deep value stocks is to use a quantitative (statistical) approach.  Carlisle explains:

Most deeply undervalued, fundamentally weak stocks are that way because their futures appear uncertain – they are losing money or marginally profitable – and, on an individual basis, don’t appear to be good candidates for purchase.  We know, however, that in aggregate they provide excellent returns, outperforming the market in the long run and suffering fewer down years than the market.  This is an area where our native intuition fails us.  As we have seen, no matter how well trained we are, humans tend to have difficulty with probabilistic, uncertain, and random processes… Since the 1950s, social scientists have been comparing the predictive abilities of traditional experts and what are known are statistical prediction rules.  The studies have found almost uniformly that statistical prediction rules are more consistently accurate than the very best experts.

I wrote about this here:  https://boolefund.com/simple-quant-models-beat-experts-in-a-wide-variety-of-areas/

The conclusion is that, for a surprisingly wide range of prediction problems – including investing – statistical prediction rules are more reliable than human experts.  Many people have objected that experts could do better than simple statistical prediction rules if they had the ability to override the rule in specific cases.  But this turns out not to be true.  The statistical prediction rules are a ceiling from which the expert detracts rather than a floor to which the expert adds.

As Daniel Kahneman explains so well in his book Thinking, Fast and Slow – see especially Part III – we humans are generally very overconfident about our ability to predict the future.  Philip Tetlock did a landmark, 20-year study of experts making political and economic predictions.  What Tetlock found based on more than 27,000 predictions over the course of two decades is that the experts were little better than chance.  See Tetlock’s Expert Political Judgment: How Good Is It? How Can We Know? (Princeton, 2005).

People, especially experts, are simply way overconfident about their ability to predict many future events.  Even Kahneman himself, after spending most of his life studying overconfidence, admits that he is “wildly overconfident” by nature, just like most people.  Overconfidence is related to many cognitive biases that people have, especially hindsight bias: https://boolefund.com/cognitive-biases/

Statistical Prediction Rules Applied to Deep Value Investing

If you don’t understand value investing – or if trailing the market for a couple of years would make you abandon a value strategy – then your best long-term investment is a low-cost broad market index fund.  Such an index fund will allow you to beat 85-90% of all investors over the course of several decades.  And it takes very little time to implement and maintain this approach.

If you understand value investing, then you should consider a quantitative value fund.  A quantitative value fund – like the Boole Microcap Fund – is a fund that systematically picks cheap stocks.  Typically, systematic stock selection is fully automated, thereby maximizing long-term results by minimizing human error.

 

BOOLE MICROCAP FUND

An equal weighted group of micro caps generally far outperforms an equal weighted (or cap-weighted) group of larger stocks over time.  See the historical chart here:  https://boolefund.com/best-performers-microcap-stocks/

This outperformance increases significantly by focusing on cheap micro caps.  Performance can be further boosted by isolating cheap microcap companies that show improving fundamentals.  We rank microcap stocks based on these and similar criteria.

There are roughly 10-20 positions in the portfolio.  The size of each position is determined by its rank.  Typically the largest position is 15-20% (at cost), while the average position is 8-10% (at cost).  Positions are held for 3 to 5 years unless a stock approaches intrinsic value sooner or an error has been discovered.

The mission of the Boole Fund is to outperform the S&P 500 Index by at least 5% per year (net of fees) over 5-year periods.  We also aim to outpace the Russell Microcap Index by at least 2% per year (net).  The Boole Fund has low fees.

 

If you are interested in finding out more, please e-mail me or leave a comment.

My e-mail: jb@boolefund.com

 

 

 

Disclosures: Past performance is not a guarantee or a reliable indicator of future results. All investments contain risk and may lose value. This material is distributed for informational purposes only. Forecasts, estimates, and certain information contained herein should not be considered as investment advice or a recommendation of any particular security, strategy or investment product. Information contained herein has been obtained from sources believed to be reliable, but not guaranteed. No part of this article may be reproduced in any form, or referred to in any other publication, without express written permission of Boole Capital, LLC.

How to Get Rich

October 9, 2022

There are many ways to get rich.  One of the best ways is through long-term investing.

A wise long-term investment for most investors is an S&P 500 index fund.  It’s just simple arithmetic, as Warren Buffett and Jack Bogle frequently observe: https://boolefund.com/warren-buffett-jack-bogle/

But you can do significantly better—roughly 18% per year (instead of 10% per year)—by systematically investing in cheap, solid microcap stocks.

Most professional investors never consider microcaps because their assets under management are too large.  Microcaps aren’t as profitable for them.  That’s why there continues to be a compelling opportunity for savvy investors.  Because microcaps are largely ignored, many get quite cheap on occasion.

Warren Buffett earned the highest returns of his career when he could invest in microcap stocks.  Buffett says he’d do the same today if he were managing small sums: https://boolefund.com/buffetts-best-microcap-cigar-butts/

Look at this summary of the CRSP Decile-Based Size and Return Data from 1927 to 2020:

Decile Market Cap-Weighted Returns Equal Weighted Returns Number of Firms (year-end 2020) Mean Firm Size (in millions)
1 9.67% 9.47% 179 145,103
2 10.68% 10.63% 173 25,405
3 11.38% 11.17% 187 12,600
4 11.53% 11.29% 203 6,807
5 12.12% 12.03% 217 4,199
6 11.75% 11.60% 255 2,771
7 12.01% 11.99% 297 1,706
8 12.03% 12.33% 387 888
9 11.55% 12.51% 471 417
10 12.41% 17.27% 1,023 99
9+10 11.71% 15.77% 1,494 199

(CRSP is the Center for Research in Security Prices at the University of Chicago.  You can find the data for various deciles here:  http://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html)

The smallest two deciles—9+10—comprise microcap stocks, which typically are stocks with market caps below $500 million.  What stands out is the equal weighted returns of the 9th and 10th size deciles from 1927 to 2020:

Microcap equal weighted returns = 15.8% per year

Large-cap equal weighted returns = ~10% per year

In practice, the annual returns from microcap stocks will be 1-2% lower because of the difficulty (due to illiquidity) of entering and exiting positions.  So we should say that an equal weighted microcap approach has returned 14% per year from 1927 to 2020, versus 10% per year for an equal weighted large-cap approach.

 

VALUE SCREEN: +2-3%

By systematically implementing a value screen—e.g., low EV/EBITDA or low P/E—to a microcap strategy, you can add 2-3% per year.

 

IMPROVING FUNDAMENTALS: +2-3%

You can further boost performance by screening for improving fundamentals.  One excellent way to do this is using the Piotroski F_Score, which works best for cheap micro caps.  See:  https://boolefund.com/joseph-piotroski-value-investing/

 

BOTTOM LINE

If you invest in microcap stocks, you can get about 14% a year.  If you also use a simple screen for value, that adds at least 2% a year.  If, in addition, you screen for improving fundamentals, that adds at least another 2% a year.  So that takes you to 18% a year, which compares quite well to the 10% a year you could get from an S&P 500 index fund.

What’s the difference between 18% a year and 10% a year?  If you invest $50,000 at 10% a year for 30 years, you end up with $872,000, which is good.  If you invest $50,000 at 18% a year for 30 years, you end up with $7.17 million, which is much better.

Please contact me if you would like to learn more.

    • My email: jb@boolefund.com.
    • My cell: 206.518.2519

 

BOOLE MICROCAP FUND

An equal weighted group of micro caps generally far outperforms an equal weighted (or cap-weighted) group of larger stocks over time.  See the historical chart here:  https://boolefund.com/best-performers-microcap-stocks/

This outperformance increases significantly by focusing on cheap micro caps.  Performance can be further boosted by isolating cheap microcap companies that show improving fundamentals.  We rank microcap stocks based on these and similar criteria.

There are roughly 10-20 positions in the portfolio.  The size of each position is determined by its rank.  Typically the largest position is 15-20% (at cost), while the average position is 8-10% (at cost).  Positions are held for 3 to 5 years unless a stock approaches intrinsic value sooner or an error has been discovered.

The mission of the Boole Fund is to outperform the S&P 500 Index by at least 5% per year (net of fees) over 5-year periods.  We also aim to outpace the Russell Microcap Index by at least 2% per year (net).  The Boole Fund has low fees.

 

 

 

 

Disclosures: Past performance is not a guarantee or a reliable indicator of future results. All investments contain risk and may lose value. This material is distributed for informational purposes only. Forecasts, estimates, and certain information contained herein should not be considered as investment advice or a recommendation of any particular security, strategy or investment product. Information contained herein has been obtained from sources believed to be reliable, but not guaranteed. No part of this article may be reproduced in any form, or referred to in any other publication, without express written permission of Boole Capital, LLC.

The Intelligent Investor

October 2, 2022

Warren Buffett, arguably the greatest investor of all time, was the only student to whom professor Ben Graham gave an A+ for the course he taught on security analysis at Columbia University.  Buffett later worked for Graham for a couple of years.  Graham was a teacher, a mentor, and a friend to Buffett.  Buffett said about Ben Graham’s The Intelligent Investor:

Chapters 8 and 20 have been the bedrock of my investing activities for more than 60 years.  I suggest that all investors read those chapters and reread them every time the market has been especially strong or weak.

(Ben Graham, by Equim43 via Wikimedia Commons)

 

CHAPTER 8: THE INVESTOR AND MARKET FLUCTUATIONS

Graham notes that stock prices fluctuate widely, and the intelligent investor should be interested in profiting from these swings.  Graham says there are two ways to do this: timing and pricing.  Timing is an attempt to predict stock prices.  Graham:

By pricing we mean the endeavor to buy stocks when they are quoted below their fair value and to sell them when they rise above such value.

Note that fair value, also called intrinsic value, is how much a knowledgeable buyer would pay for a business, based either upon how much the business can earn or based upon the balance sheet of the business.

Graham continues by noting that trying to time the market is speculation, and will not result in profits over the long term.  Many investors feel compelled to listen to stock market forecasts.  But, says Graham:

There is no basis either in logic or in experience for assuming that any typical or average investor can anticipate market movements more successfully than the general public, of which he is himself a part.

This is understating the case.  There is no evidence that any individual forecaster has been able to consistently predict the short-term movements of the stock market.

Illustration by Maxim Popov

Graham points out that one factor that leads the speculator to rely on shorter-term stock market predictions is that the speculator is in a hurry to make money.  An investor, by contrast, does not mind waiting one year or several years for a sound investment to pay off.

Furthermore, timing formulas based on the past tend not to work as well in the future, as Graham explains:

Those formulas that gain adherents and importance do so because they have worked well over a period, or sometimes merely because they have been plausibly adapted to the statistical record of the past.  But as their acceptance increases, their reliability tends to diminish.  This happens for two reasons: First, the passage of time brings new conditions which the old formula no longer fits.  Second, in stock-market affairs the popularity of a trading theory has itself an influence on the market’s behavior which detracts in the long run from its profit-making possibilities.

Graham writes of several theories that tried to identify buying conditions and selling conditions in the market.  Even Graham himself developed a “central value method.”  However, says Graham:

The moral seems to be that any approach to moneymaking in the stock market which can be easily described and followed by a lot of people is by its terms too simple and too easy to last.

An an investor, one should expect wide fluctuations in stock prices.  But what do you do after stock prices have advanced a great deal?  Graham:

But has the price risen too high, and should you think of selling?  Or should you kick yourself for not having bought more shares when the level was lower?  Orworst thought of allshould you now give way to the bull-market atmosphere, become infected with the enthusiasm, the overconfidence and the greed of the great public (of which, after all, you are a part), and make larger and dangerous commitments?  Presented thus in print, the answer to the last question is a self-evident no, but even the intelligent investor is likely to need considerable will power to keep from following the crowd.

Graham continues:

It is for these reasons of human nature, even more than by calculation of financial gain or loss, that we favor some kind of mechanical method for varying the proportion of bonds to stocks in the investor’s portfolio.  The chief advantage, perhaps, is that such a formula will give him something to do.  As the market advances he will from time to time make sales out of his stockholdings, putting the proceeds into bonds; as it declines he will reverse the procedure.  These activities will provide him some outlet for his otherwise too-pent-up energies.  If he is the right kind of investor he will take added satisfaction from the thought that his operations are exactly opposite from those of the crowd.

Business Valuations versus Stock-Market Valuations

Graham writes:

The impact of market fluctuations upon the investor’s true situation may be considered also from the standpoint of the shareholder as the part owner of various businesses.  The holder of marketable shares actually has a double status, and with it the privilege of taking advantage of either at his choice.  On the one hand his position is analogous to that of a minority shareholder or silent partner in a private business.  Here his results are entirely dependent on the profits of the enterprise or on a change in the underlying value of its assets.  He would usually determine the value of such a private-business interest by calculating his share of the net worth as shown in the most recent balance sheet.  On the other hand, the common-stock investor holds a piece of paper, an engraved stock certificate, which can be sold in a matter of minutes at a price which varies from moment to momentwhen the market is open, that isand often is far removed from the balance-sheet value.

Graham goes on to note that many successful businesses sell above their net worth.  (Note that net worth is also called book value, balance-sheet value, asset value, net asset value, and tangible book value.)  In this sense, Graham considers such businesses speculative as compared to unspectacular businesses selling at book value or below.  (It should be noted that businesses with a sustainably high ROICreturn on invested capitalshould sell above book value.  Some successful technology companies fall into this category.)

Graham:

The previous discussion leads us to a conclusion of practical importance to the conservative investor in common stocks.  If he is to pay some special attention to the selection of his portfolio, it might be best for him to concentrate on issues selling at a reasonably close approximation to their tangible-asset valuesay, at not more than one-third above that figure.  Purchases made at such levels, or lower, may with logic be regarded as related to the company’s balance sheet, and as having a justification or support independent of the fluctuating market prices….

(Illustration by Teguh Jati Prasetyo)

Graham adds:

A caution is needed here.  A stock does not become a sound investment merely because it can be bought at close to its asset value.  The investor should demand, in addition, a satisfactory ratio of earnings to price, a sufficiently strong financial position, and the prospect that its earnings will at least be maintained over the years.  This may appear like demanding a lot from a modestly priced stock, but the prescription is not hard to fill under all but dangerously high market conditions.  Once the investor is willing to forgo brilliant prospectsi.e., better than average expected growthhe will have no difficulty in finding a wide selection of issues meeting these criteria.

Graham then makes a central point:

The investor with a stock portfolio having such book values behind it can take a much more independent and detached view of stock-market fluctuations than those who have paid high multiples of both earnings and tangible assets.  As long as the earning power of his holdings remains satisfactory, he can give as little attention as he pleases to the vagaries of the stock market.  More than that, at times he can use these vagaries to play the master game of buying low and selling high.

The A. & P. Example

Graham gives the example of the Great Atlantic & Pacific Tea Co.  The shares sold as high as $494 in 1929, and ended up declining to a new low of $36 in 1938.  At that price, the company had a market capitalization of  of $126 million, which was lower than its net current assets of $134 million.  Essentially, the company was selling below net cash, which is cash minus all liabilities.  So its value as a going concern was lower than its value in a liquidation would be.  Graham explains:

Why?  First, because there were threats of special taxes on chain stores; second, because net profits had fallen off in the previous year; and, third, because the general market was depressed.  The first of these reasons was an exaggerated and eventually groundless fear; the other two were typical of temporary influences.

What about the investor who bought at $80 in 1937?  Graham says the investor should carefully have studied the situation, but should have concluded that the market price was a temporary vagary.  In fact, the investor should have bought more if he had the funds and the courage to do so.

By 1939, A. & P. was selling at $117.5.  In the years following 1949, A. & P. continued to advance, eventually reaching a split-adjusted price of $705.  At that price, the stock had a price-to-earnings ratio of 30, which implied that holders of the stock expected brilliant growth.  Such expectations were not justified.  The stock fell to an equivalent of $340, but even then was still not a bargain.  Eventually if fell to the equivalent of $215 in 1970 and then $180 in 1972, when the company reported its first quarterly loss in its history.

Graham comments:

We see in this history how wide can be the vicissitudes of a major American enterprise in little more than a single generation, and also with what miscalculations and excesses of optimism and pessimism the public has valued its shares.

Graham concludes:

There are two chief morals to this story.  The first is that the stock market often goes far wrong, and sometimes an alert and courageous investor can take advantage of its patent errors.  The other is that most businesses change in character and quality over the years, sometimes for the better, perhaps more often for the worst.  The investor need not watch his companies’ performance like a hawk; but he should give it a good, hard look from time to time.

Graham returns to the idea that a holder of marketable shares is like someone who owns a private business:

The true investor scarcely ever is forced to sell his shares, and at all other times he is free to disregard the current price quotation.  He need pay attention to it and act upon it only to the extent that it suits his book, and no more.  Thus the investor who permits himself to be stampeded or unduly worried by unjustified market declines in his holdings is perversely transforming his basic advantage into a basic disadvantage.  That man would be better off if his stocks had no market quotation at all, for he would then be spared the mental anguish caused him by other persons’ mistakes of judgment.

Graham then introduces the concept of “Mr. Market.”  Imagine you own a share in a private business:

One of your partners, named Mr. Market, is very obliging indeed.  Every day he tells you what he thinks your interest is worth and furthermore offers either to buy you out or to sell you an additional interest on that basis.  Sometimes his idea of value appears plausible and justified by business developments and prospects as you know them.  Often, on the other hand, Mr. Market lets his enthusiasm or his fears run away with him, and the value he proposes seems to you a little short of silly.

Graham makes the point that you do not form your opinion about the value of the business based on Mr. Market’s daily communication:

Only in case you agree with him, or in case you want to trade with him.  You may be happy to sell out to him when he quotes you a ridiculously high price, and equally happy to buy from him when his price is low.  But the rest of the time you will be wiser to form your own ideas of the value of your holdings, based on full reports from the company about its operations and financial position.

Graham argues that owning a share of stock is similar to the Mr. Market analogy:

Basically, price fluctuations have only one significant meaning for the true investor.  They provide him with an opportunity to buy wisely when prices fall sharply and to sell wisely when they advance a great deal.  At other times he will do better if he forgets about the stock market and pays attention to his dividend returns and to the operating results of his companies.

(Illustration by Andrii Vinnikov)

Graham concludes:

The investor with a portfolio of sound stocks should expect the prices to fluctuate and should neither be concerned by sizable declines nor become excited by sizable advances.  He should always remember that market quotations are there for his convenience, either to be taken advantage of or to be ignored.  He should never buy a stock because it has gone up or sell one because it has gone down.

Graham makes one final point:

Good managements produce a good average market price, and bad managements produce bad market prices.

 

CHAPTER 20: “MARGIN OF SAFETY” AS THE CENTRAL CONCEPT OF INVESTMENT

Graham writes:

In the old legend the wise men finally boiled down the history of mortal affairs into the single phrase, “This too will pass.”  Confronted with a like challenge to distill the secret of sound investment into three words, we venture the motto, MARGIN OF SAFETY.

Graham comments:

Here the function of the margin of safety is, in essence, that of rendering unnecessary an accurate estimate of the future.  If the margin is a large one, then it is enough to assume that future earnings will not fall far below those of the past in order for an investor to feel sufficiently protected against the  vicissitudes of time.

Graham again:

The margin of safety is always dependent on the price paid.  It will be large at one price, small at some higher price, nonexistent at some still higher price.

(Photo by Chuahtc8)

Graham explains margin of safety:

The margin-of-safety idea becomes much more evident when we apply it to the field of undervalued or bargain securities.  We have here, by definition, a favorable difference between price on the one hand and indicated or appraised value on the other.  That difference is the margin of safety.  It is available for absorbing the effect of miscalculations or worse than average luck.  The buyer of bargain issues places particular emphasis on the ability of the investment to withstand adverse developments.  For in most such cases he has no real enthusiasm about the company’s prospects.

Graham goes on to note that a moderate decline in earnings power will not necessarily prevent a cheaply bought stock from producing a satisfactory investment result.

A Criterion of Investment versus Speculation

Graham writes:

Probably most speculators believe they have the odds in their favor when they take their chances, and therefore they may lay claim to a safety margin in their proceedings… But such claims are unconvincing.  They rest on subjective judgment, unsupported by any body of favorable evidence or any conclusive line of reasoning.  We greatly doubt whether the man who stakes money on his view that the market is heading up or down can ever be said to be protected by a margin of safety in any useful sense of the phrase.

Graham observes that, by contrast, buying stocks below a conservative appraisal of intrinsic value does imply a margin of safety and therefore does classify as investment rather than speculation.

To Sum Up

Graham sums up the chapter:

Investment is most intelligent when it is most businesslike.  It is amazing to see how many capable businessmen try to operate in Wall Street with complete disregard of all the sound principles through which they have gained success in their own undertakings.  Yet every corporate security may best be viewed, in the first instance, as an ownership interest in, or a claim against, a specific business enterprise.  And if a person sets out to make profits from security purchases and sales, he is embarking on a business venture of his own, which must be run in accordance with accepted business principles if it is to have a chance of success.

Graham says the investor should know as much about the intrinsic values of the businesses in which he invests as he would need to know about the value of merchandise that he proposed to manufacture and sell.

Graham adds that the investor should be able to supervise adequately the running of the business in which he invests, and the investor should have confidence in the integrity and ability of the managers running the business.

Moreover, the investor should have a reasonable chance of profit, while possible losses should be minimal by comparison.

Furthermore, notes Graham, have the courage of your knowledge and experience, regardless of how many others agree or disagree.

You are neither right nor wrong because the crowd disagrees with you.  You are right because your data and reasoning are right… in the world of securities, courage becomes the supreme virtue after adequate knowledge and a tested judgment are at hand.

 

BOOLE MICROCAP FUND

An equal weighted group of micro caps generally far outperforms an equal weighted (or cap-weighted) group of larger stocks over time.  See the historical chart here:  https://boolefund.com/best-performers-microcap-stocks/

This outperformance increases significantly by focusing on cheap micro caps.  Performance can be further boosted by isolating cheap microcap companies that show improving fundamentals.  We rank microcap stocks based on these and similar criteria.

There are roughly 10-20 positions in the portfolio.  The size of each position is determined by its rank.  Typically the largest position is 15-20% (at cost), while the average position is 8-10% (at cost).  Positions are held for 3 to 5 years unless a stock approaches intrinsic value sooner or an error has been discovered.

The mission of the Boole Fund is to outperform the S&P 500 Index by at least 5% per year (net of fees) over 5-year periods.  We also aim to outpace the Russell Microcap Index by at least 2% per year (net).  The Boole Fund has low fees.

 

If you are interested in finding out more, please e-mail me or leave a comment.

My e-mail: jb@boolefund.com

 

 

 

Disclosures: Past performance is not a guarantee or a reliable indicator of future results. All investments contain risk and may lose value. This material is distributed for informational purposes only. Forecasts, estimates, and certain information contained herein should not be considered as investment advice or a recommendation of any particular security, strategy or investment product. Information contained herein has been obtained from sources believed to be reliable, but not guaranteed. No part of this article may be reproduced in any form, or referred to in any other publication, without express written permission of Boole Capital, LLC.

The Black Swan

September 25, 2022

Nassim Nicholas Taleb  is the author of several books, including Fooled by RandomnessThe Black Swan, and Antifragile.  I wrote about Fooled by Randomness here: https://boolefund.com/fooled-by-randomness/

Today’s blog post is a summary of Taleb’s The Black Swan.  If you’re an investor, or if you have any interest in predictions or in history, then this is a MUST-READ book.  One of Taleb’s main points is that Black Swans, which are unpredictable, can be either positive or negative.  It’s crucial to try to be prepared for negative Black Swans and to try to benefit from positive Black Swans.  However, many measurements of risk in finance assume a statistical distribution that is normal when they should assume a distribution that is fat-tailed.  These standard measures of risk won’t prepare you for a Black Swan.

That said, Taleb is an option trader, whereas I am a value investor.  For me, if you buy a stock far below liquidation value, then usually you have a margin of safety.  A group of such stocks will outperform the market over time while carrying low risk.  Furthermore, if you’re a long-term investor, then you can either adopt a value investing approach or you can simply invest in low-cost index funds.  Either way, given a long enough period of time, you should get good results.  The market has recovered from every crash and has eventually gone on to new highs.  Yet Taleb misses this point.

Nonetheless, although Taleb overlooks value investing and index funds, his views on predictions and on history are very insightful and should be studied by every thinking person.

Black Swan in Auckland, New Zealand.  Photo by Angela Gibson.

Here’s the outline:

    • Prologue

PART ONE: UMBERTO ECO’S ANTILIBRARY, OR HOW WE SEEK VALIDATION

    • Chapter 1: The Apprenticeship of an Empirical Skeptic
    • Chapter 2: Yevgenia’s Black Swan
    • Chapter 3: The Speculator and the Prostitute
    • Chapter 4: One Thousand and One Days, or How Not to Be a Sucker
    • Chapter 5: Confirmation Schmonfirmation!
    • Chapter 6: The Narrative Fallacy
    • Chapter 7: Living in the Antechamber of Hope
    • Chapter 8: Giacomo Casanova’s Unfailing Luck: The Problem of Silent Evidence
    • Chapter 9: The Ludic Fallacy, or the Uncertainty of the Nerd

PART TWO: WE JUST CAN’T PREDICT

    • Chapter 10: The Scandal of Prediction
    • Chapter 11: How to Look for Bird Poop
    • Chapter 12: Epistemocracy, a Dream
    • Chapter 13: Apelles the Painter, or What Do You Do if You Cannot Predict?

PART THREE: THOSE GRAY SWANS OF EXTREMISTAN

    • Chapter 14: From Mediocristan to Extremistan, and Back
    • Chapter 15: The Bell Curve, That Great Intellectual Fraud
    • Chapter 16: The Aesthetics of Randomness
    • Chapter 17: Locke’s Madmen, or Bell Curves in the Wrong Places
    • Chapter 18: The Uncertainty of the Phony

PART FOUR: THE END

    • Chapter 19: Half and Half, or How to Get Even with the Black Swan

 

PROLOGUE

Taleb writes:

Before the discovery of Australia, people in the Old World were convinced that all swans were white, an unassailable belief as it seemed completely confirmed by empirical evidence… It illustrates a severe limitation to our learning from observations or experience and the fragility of our knowledge.  One single observation can invalidate a general statement derived from millenia of confirmatory sightings of millions of white swans.

Taleb defines a black swan as having three attributes:

    • First, it is an outlier, as it lies outside the realm of regular expectations, because nothing in the past can convincingly point to its possibility.
    • Second, it carries an extreme impact.
    • Third, in spite of its outlier status, human nature makes us concoct explanations for its occurrence after the fact, making it explainable and predictable.

Taleb notes that the effect of Black Swans has been increasing in recent centuries.  Furthermore, social scientists still assume that risks can be modeled using the normal distribution, i.e., the bell curve.  Social scientists have not incorporated “Fat Tails” into their assumptions about risk.  (A fat-tailed statistical distribution, as compared to a normal distribution, carries higher probabilities for extreme outliers.)

(Illustration by Peter Hermes Furian:  The red curve is a normal distribution, whereas the orange curve has fat tails.)

Taleb continues:

Black Swan logic makes what you don’t know far more relevant than what you do know.  Consider that many Black Swans can be caused and exacerbated by their being unexpected.

Taleb mentions the Sept. 11, 2001 terrorist attack on the twin towers.  If such an attack had been expected, then it would have been prevented.  Taleb:

Isn’t it strange to see an event happening precisely because it was not supposed to happen?  What kind of defense do we have against that? … It may be odd that, in such a strategic game, what you know can be truly inconsequential.

Taleb argues that Black Swan logic applies to many areas in business and also to scientific theories.  Taleb makes a general point about history:

The inability to predict outliers implies the inability to predict the course of history, given the share of these events in the dynamics of events.

Indeed, people, especially experts, have a terrible record in forecasting political and economic events.  Taleb advises:

Black Swans being unpredictable, we need to adjust to their existence (rather than naively try to predict them).  There are so many things we can do if we focus on antiknowledge, or what we do not know.  Among many other benefits, you can set yourself up to collect serendipitous Black Swans (of the positive kind) by maximizing your exposure to them.  Indeed, in some domains—such as scientific discovery and venture capital investments—there is a disproportionate payoff from the unknown, since you typically have little to lose and plenty to gain from a rare event… The strategy is, then, to tinker as much as possible and try to collect as many Black Swan opportunities as you can.

Taleb introduces the terms Platonicity and the Platonic fold:

Platonicity is what makes us think that we understand more than we actually do.  But this does not happen everywhere.  I am not saying that Platonic forms don’t exist.  Models and constructions, these intellectual maps of reality, are not always wrong; they are wrong only in some specific applications.  The difficulty is that a) you do not know before hand (only after the fact) where the map will be wrong, and b) the mistakes can lead to severe consequences…

The Platonic fold is the explosive boundary where the Platonic mindset enters in contact with messy reality, where the gap between what you know and what you think you know becomes dangerously wide.  It is here that the Black Swan is produced.

 

PART ONE: UMBERTO ECO’S ANTILIBRARY, OR HOW WE SEEK VALIDATION

Umberto Eco’s personal library contains thirty thousand books.  But what’s important are the books he has not yet read.  Taleb:

Read books are far less valuable than unread books.  The library should contain as much of what you do not know as your financial means, mortgage rates, and the currently tight real-estate market allow you to put there… Indeed, the more you know, the larger the rows of unread books.  Let us call this collection of unread books an antilibrary.

Taleb adds:

Let us call an antischolar—someone who focuses on the unread books, and makes an attempt not to treat his knowledge as a treasure, or even a possession, or even a self-esteem enhancement device—a skeptical empiricist.

(Photo by Pp1)

 

CHAPTER 1: THE APPRENTICESHIP OF AN EMPIRICAL SKEPTIC

Taleb says his family is from “the Greco-Syrian community, the last Byzantine outpost in northern Syria, which included what is now called Lebanon.”  Taleb writes:

People felt connected to everything they felt was worth connecting to; the place was exceedingly open to the world, with a vastly sophisticated lifestyle, a prosperous economy, and temperate weather just like California, with snow-covered mountains jutting above the Mediterranean.  It attracted a collection of spies (both Soviet and Western), prostitutes (blondes), writers, poets, drug dealers, adventurers, compulsive gamblers, tennis players, apres-skiers, and merchants—all professions that complement one another.

Taleb writes about when he was a teenager.  He was a “rebellious idealist” with an “ascetic taste.”  Taleb:

As a teenager, I could not wait to go settle in a metropolis with fewer James Bond types around.  Yet I recall something that felt special in the intellectual air.  I attended the French lycee that had one of the highest success rates for the French baccalaureat (the high school degree), even in the subject of the French language.  French was spoken there with some purity: as in prerevolutionary Russia, the Levantine Christian and Jewish patrician class (from Istanbul to Alexandria) spoke and wrote formal French as a language of distinction.  The most privileged were sent to school in France, as both my grandfathers were… Two thousand years earlier, by the same instinct of linguistic distinction, the snobbish Levantine patricians wrote in Greek, not the vernacular Aramaic… And, after Hellenism declined, they took up Arabic.  So in addition to being called a “paradise,” the place was also said to be a miraculous crossroads of what are superficially tagged “Eastern” and “Western” cultures.

Then a Black Swan hit:

The Lebanese “paradise” suddenly evaporated, after a few bullets and mortar shells… after close to thirteen centuries of remarkable ethnic coexistence, a Black Swan, coming out of nowhere, transformed the place from heaven to hell.  A fierce civil war began between Christians and Moslems, including the Palastinian refugees who took the Moslem side.  It was brutal, since the combat zones were in the center of town and most of the fighting took place in residential areas (my high school was only a few hundred feet from the war zone).  The conflict lasted more than a decade and a half.

Taleb makes a general point about history:

The human mind suffers from three ailments as it comes into contact with history, what I call the triplet of opacity.  They are:

    • the illusion of understanding, or how everyone thinks he knows what is going on in a world that is more complicated (or random) than they realize;
    • the retrospective distortion, or how we can assess matters only after the fact, as if they were in a rearview mirror (history seems clearer and more organized in history books than in empirical reality); and
    • the overvaluation of factual information and the handicap of authoritative and learned people, particularly when they create categories—when they “Platonify.”

Taleb points out that a diary is a good way to record events as they are happening.  This can help later to put events in their context.

Photo by Anton Samsonov

Taleb writes about the danger of oversimplification:

Any reduction of the world around us can have explosive consequences since it rules out some sources of uncertainty; it drives us to a misunderstanding of the fabric of the world.  For instance, you may think that radical Islam (and its values) are your allies against the threat of Communism, and so you may help them develop, until they send two planes into downtown Manhattan.

 

CHAPTER 2: YEVGENIA’S BLACK SWAN

Taleb:

Five years ago, Yevgenia Nikolayevna Krasnova was an obscure and unpublished novelist, with an unusual background.  She was a neuroscientist with an interest in philosophy (her first three husbands had been philosophers), and she got it into her stubborn Franco-Russian head to express her research and ideas in literary form.

Most publishers largely ignored Yevgenia.  Publishers who did look at Yevnegia’s book were confused because she couldn’t seem to answer the most basic questions.  “Is this fiction or nonfiction?”  “Who is this book written for?”  (Five years ago, Yevgenia attended a famous writing workshop.  The instructor told her that her case was hopeless.)

Eventually the owner of a small unknown publishing house agreed to publish Yevgenia’s book.  Taleb:

It took five years for Yevnegia to graduate from the “egomaniac without anything to justify it, stubborn and difficult to deal with” category to “persevering, resolute, painstaking, and fiercely independent.”  For her book slowly caught fire, becoming one of the great and strange successes in literary history, selling millions of copies and drawing so-called critical acclaim…

Yevgenia’s book is a Black Swan.

 

 

CHAPTER 3: THE SPECULATOR AND THE PROSTITUTE

Taleb introduces Mediocristan and Extremistan:

Mediocristan Extremistan
Nonscalable Scalable
Mild or type 1 randomness Wild (even superwild) type 2 randomness
The most typical member is mediocre The most “typical” is either giant or dwarf, i.e., there is no typical member
Winners get a small segment of the total pie Winner-take-almost-all effects
Example: Audience of an opera singer before the gramophone Today’s audience for an artist
More likely to be found in our ancestral environment More likely to be found in our modern environment
Impervious to the Black Swan Vulnerable to the Black Swan
Subject to gravity There are no physical constraints on what a number can be
Corresponds (generally) to physical quantities, i.e., height Corresponds to numbers, say, wealth
As close to utopian equality as reality can spontaneously deliver Dominated by extreme winner-take-all inequality
Total is not determined by a single instance or observation Total will be determined by a small number of extreme events
When you observe for a while you can get to know what’s going on It takes a long time to get to know what’s going on
Tyranny of the collective Tyranny of the accidental
Easy to predict from what you see and extend to what you do not see Hard to predict from past information
History crawls History makes jumps
Events are distributed according to the “bell curve” or its variations The distribution is either Mandelbrotian “gray” Swans (tractable scientifically) or totally intractable Black Swans

Taleb observes that Yevgenia’s rise from “the second basement to superstar” is only possible in Extremistan.

(Photo by Flavijus)

Taleb comments on knowledge and Extremistan:

What you can know from data in Mediocristan augments very rapidly with the supply of information.  But knowledge in Extremistan grows slowly and erratically with the addition of data, some of it extreme, possibly at an unknown rate.

Taleb gives many examples:

Matters that seem to belong to Mediocristan (subjected to what we call type 1 randomness): height, weight, calorie consumption, income for a baker, a small restaurant owner, a prostitute, or an orthodontist; gambling profits (in the very special case, assuming the person goes to a casino and maintains a constant betting size), car accidents, mortality rates, “IQ” (as measured).

Matters that seem to belong to Extremistan (subjected to what we call type 2 randomness): wealth, income, book sales per author, book citations per author, name recognition as a “celebrity,” number of references on Google, populations of cities, uses of words in a vocabulary, numbers of speakers per language, damage caused by earthquakes, deaths in war, deaths from terrorist incidents, sizes of planets, sizes of companies, stock ownership, height between species (consider elephants and mice), financial markets (but your investment manager does not know it), commodity prices, inflation rates, economic data.  The Extremistan list is much longer than the prior one.

Taleb concludes the chapter by introducing “gray” swans, which are rare and consequential, but somewhat predictable:

They are near-Black Swans.  They are somewhat tractable scientifically—knowing about their incidence should lower your surprise; these events are rare but expected.  I call this special case of “gray” swans Mandelbrotian randomness.  This category encompasses the randomness that produces phenomena commonly known by terms such as scalable, scale-invariant, power laws, Pareto-Zipf laws, Yule’s law, Paretian-stable processes, Levy-stable, and fractal laws, and we will leave them aside for now since they will be covered in some depth in Part Three…

You can still experience severe Black Swans in Mediocristan, though not easily.  How?  You may forget that something is random, think that it is deterministic, then have a surprise.  Or you can tunnel and miss on a source of uncertainty, whether mild or wild, owing to lack of imagination—most Black Swans result from this “tunneling” disease, which I will discuss in Chapter 9.

 

CHAPTER 4: ONE THOUSAND AND ONE DAYS, OR HOW NOT TO BE A SUCKER

Photo of turkey by Chris Galbraith

Taleb introduces the Problem of Induction by using an example from the philosopher Bertrand Russell:

How can we logically go from specific instances to reach general conclusions?  How do we know what we know?  How do we know that what we have observed from given objects and events suffices to enable us to figure out their other properties?  There are traps built into any kind of knowledge gained from observation.

Consider a turkey that is fed every day.  Every single feeding will firm up the bird’s belief that it is a general rule of life to be fed every day by friendly members of the human race “looking out for its best interests,” as a politician would say.  On the afternoon of the Wednesday before Thanksgiving, something unexpected will happen to the turkey.  It will incur a revision of belief.

The rest of this chapter will outline the Black Swan problem in its original form: How can we know the future, given knowledge of the past; or, more generally, how can we figure out properties of the (infinite) unknown based on the (finite) known?

Taleb says that, as in the example of the turkey, the past may be worse than irrelevant.  The past may be “viciously misleading.”  The turkey’s feeling of safety reached its high point just when the risk was greatest.

Roasted turkey.  Photo by Alexander Raths.

Taleb gives the example of banking, which was seen and presented as “conservative,” based on the rarity of loans going bust.  However, you have to look at the loans over a very long period of time in order to see if a given bank is truly conservative.  Taleb:

In the summer of 1982, large American banks lost close to all their past earnings (cumulatively), about everything they ever made in the history of American banking—everything.  They had been lending to South and Central American countries that all defaulted at the same time—”an event of an exceptional nature”… They are not conservative; just phenomenally skilled at self-deception by burying the possibility of a large, devastating loss under the rug.  In fact, the travesty repeated itself a decade later, with the “risk-conscious” large banks once again under financial strain, many of them near-bankrupt, after the real-estate collapse of the early 1990s in which the now defunct savings and loan industry required a taxpayer-funded bailout of more than half a trillion dollars.

Taleb offers another example: the hedge fund Long-Term Capital Management (LTCM).  The fund calculated risk using the methods of two Nobel Prize-winning economists.  According to these calculations, risk of blowing up was infinitesimally small.  But in 1998, LTCM went bankrupt almost instantly.

A Black Swan is always relative to your expectations.  LTCM used science to create a Black Swan.

Taleb writes:

In general, positive Black Swans take time to show their effect while negative ones happen very quickly—it is much easier and much faster to destroy than to build.

Although the problem of induction is often called “Hume’s problem,” after the Scottish philosopher and skeptic David Hume, Taleb holds that the problem is older:

The violently antiacademic writer, and antidogma activist, Sextus Empiricus operated close to a millenium and a half before Hume, and formulated the turkey problem with great precision… We surmise that he lived in Alexandria in the second century of our era.  He belonged to a school of medicine called “empirical,” since its practitioners doubted theories and causality and relied on past experience as guidance in their treatment, though not putting much trust in it.  Furthermore, they did not trust that anatomy revealed function too obviously…

Sextus represented and jotted down the ideas of the school of the Pyrrhonian skeptics who were after some form of intellectual therapy resulting from the suspension of belief… The Pyrrhonian skeptics were docile citizens who followed customs and traditions whenever possible, but taught themselves to systematically doubt everything, and thus attain a level of serenity.  But while conservative in their habits, they were rabid in their fight against dogma.

Taleb asserts that his main aim is how not to be a turkey.

In a way, all I care about is making a decision without being the turkey.

Taleb introduces the themes for the next five chapters:

    • We focus on preselected segments of the seen and generalize from it to the unseen: the error of confirmation.
    • We fool ourselves with stories that cater to our Platonic thirst for distinct patterns: the narrative fallacy.
    • We behave as if the Black Swan does not exist: human nature is not programmed for Black Swans.
    • What we see is not necessarily all that is there.  History hides Black Swans from us and gives us a mistaken idea about the odds of these events: this is the distortion of silent evidence.
    • We “tunnel”: that is, we focus on a few well-defined sources of uncertainty, on too specific a list of Black Swans (at the expense of the others that do not easily come to mind).

 

CHAPTER 5: CONFIRMATION SHMONFIRMATION!

Taleb asks about two hypothetical situations.  First, he had lunch with O.J. Simpson and O.J. did not kill anyone during the lunch.  Isn’t that evidence that O.J. Simpson is not a killer?  Second, Taleb imagines that he took a nap on the railroad track in New Rochelle, New York.  He didn’t die during his nap, so isn’t that evidence that it’s perfectly safe to sleep on railroad tracks?  Of course, both of these situations are analogous to the 1,001 days during which the turkey was regularly fed.  Couldn’t the turkey conclude that there’s no evidence of any sort of Black Swan?

The problem is that people confuse no evidence of Black Swans with evidence of no possible Black Swans.  Just because there has been no evidence yet of any possible Black Swans does not mean that there’s evidence of no possible Black Swans.  Taleb calls this confusion the round-trip fallacy, since the two statements are not interchangeable.

Taleb writes that our minds routinely simplify matters, usually without our being consciously aware of it.  Note: In his book, Thinking, Fast and Slow, the psychologist Daniel Kahneman argues that System 1, our intuitive system, routinely oversimplifies, usually without our being consciously aware of it.

Taleb continues:

Many people confuse the statement “almost all terrorists are Moslems” with “almost all Moslems are terrorists.”  Assume that the first statement is true, that 99 percent of terrorists are Moslems.  This would mean that only about .001 percent of Moslems are terrorists, since there are more than one billion Moslems and only, say, ten thousand terrorists, one in a hundred thousand.  So the logical mistake makes you (unconsciously) overestimate the odds of a randomly drawn individual Moslem person… being a terrorist by close to fifty thousand times!

Taleb comments:

Knowledge, even when it is exact, does not often lead to appropriate actions because we tend to forget what we know, or forget how to process it properly if we do not pay attention, even when we are experts.

Taleb notes that the psychologists Daniel Kahneman and Amos Tversky did a number of experiments in which they asked professional statisticians statistical questions not phrased as statistical questions.  Many of these experts consistently gave incorrect answers.

Taleb explains:

This domain specificity of our inferences and reactions works both ways: some problems we can understand in their applications but not in textbooks; others we are better at capturing in the textbook than in the practical application.  People can manage to effortlessly solve a problem in a social situation but struggle when it is presented as an abstract logical problem.  We tend to use different mental machinery—so-called modules—in different situations: our brain lacks a central all-purpose computer that starts with logical rules and applies them equally to all possible situations.

Note: Again, refer to Daniel Kahneman’s book, Thinking, Fast and Slow.  System 1 is the fast-thinking intuitive system that works effortlessly and often subconsciously.  System 1 is often right, but sometimes very wrong.  System 2 is the logical-mathematical system that can be trained to do logical and mathematical problems.  System 2 is generally slow and effortful, and we’re fully conscious of what System 2 is doing because we have to focus our attention for it to operate. See: https://boolefund.com/cognitive-biases/

Taleb next writes:

An acronym used in the medical literature is NED, which stands for No Evidence of Disease.  There is no such thing as END, Evidence of No Disease.  Yet my experience discussing this matter with plenty of doctors, even those who publish papers on their results, is that many slip into the round-trip fallacy during conversation.

Doctors in the midst of the scientific arrogance of the 1960s looked down at mothers’ milk as something primitive, as if it could be replicated by their laboratories—not realizing that mothers’ milk might include useful components that could have eluded their scientific understanding—a simple confusion of absence of evidence of the benefits of mothers’ milk with evidence of absence of the benefits (another case of Platonicity as “it did not make sense” to breast-feed when we could simply use bottles).  Many people paid the price for this naive inference: those who were not breast-fed as infants turned out to be at an increased risk of a collection of health problems, including a higher likelihood of developing certain types of cancer—there had to be in mothers’ milk some necessary nutrients that still elude us.  Furthermore, benefits to mothers who breast-feed were also neglected, such as a reduction in the risk of breast cancer.

Taleb makes the following point:

I am not saying here that doctors should not have beliefs, only that some kinds of definitive, closed beliefs need to be avoided… Medicine has gotten better—but many kinds of knowledge have not.

Taleb defines naive empiricism:

By a mental mechanism I call naive empiricism, we have a natural tendency to look for instances that confirm our story and our vision of the world—these instances are always easy to find…

Taleb makes an important point here:

Even in testing a hypothesis, we tend to look for instances where the hypothesis proved true.

Daniel Kahneman has made the same point.  System 1 (intuition) automatically looks for confirmatory evidence, but even System 2 (the logical-mathematical-rational system) naturally looks for evidence that confirms a given hypothesis.  We have to train System 2 not only to do logic and math, but also to look for disconfirming rather than confirming evidence.  Taleb says:

We can get closer to the truth by negative instances, not by verification!  It is misleading to build a general rule from observed facts.  Contrary to conventional wisdom, our body of knowledge does not increase from a series of confirmatory observations, like the turkey’s.

Taleb adds:

Sometimes a lot of data can be meaningless; at other times one single piece of information can be very meaningful.  It is true that a thousand days cannot prove you right, but one day can prove you to be wrong.

Taleb introduces the philosopher Karl Popper and his method of conjectures and refutations.  First you develop a conjecture (hypothesis).  Then you focus on trying to refute the hypothesis.  Taleb:

If you think the task is easy, you will be disappointed—few humans have a natural ability to do this.  I confess that I am not one of them; it does not come naturally to me.

Our natural tendency, whether using System 1 or System 2, is to look only for corroboration.  This is called confirmation bias.

Illustration by intheskies

There are exceptions, notes Taleb.  Chess grand masters tend to look at where their move might be weak, whereas rookie chess players only look for confirmation.  Similarly, George Soros developed a unique ability to look always for evidence that his current hypothesis is wrong.  As a result of this and not getting attached to his opinions, Soros quickly exited many of his trades that wouldn’t have worked.  Soros is one of the most successful macro investors ever.

Taleb observes that seeing a red mini Cooper actually confirms the statement that all swans are white.  Why?  Because if all swans are white, then all nonwhite objects are not swans; in other words, the statement “if it’s a swan, then it’s white” is logically equivalent to the statement “if it’s not white, then it’s not a swan” (since all swans are white).  Taleb:

This argument, known as Hempel’s raven paradox, was rediscovered by my friend the (thinking) mathematician Bruno Dupire during one of our intense meditating walks in London—one of those intense walk-discussions, intense to the point of our not noticing the rain.  He pointed to a red Mini and shouted, “Look, Nassim, look!  No Black Swan!”

Again: Finding instances that confirm the statement “if it’s not white, then it’s not a swan” is logically equivalent to finding instances that confirm the statement “if it’s a swan, then it’s white.”  So consider all the objects that confirm the statement “if it’s not white, then it’s not a swan”:  red Mini’s, gray clouds, green cucumbers, yellow lemons, brown soil, etc.  The paradox is that we seem to gain ever more information about swans by looking at an infinite series of nonwhite objects.

Taleb concludes the chapter by noting that our brains evolved to deal with a much more primitive environment than what exists today, which is far more complex.

…the sources of Black Swans today have multiplied beyond measurability.  In the primitive environment they were limited to newly encountered wild animals, new enemies, and abrupt weather changes.  These events were repeatable enough for us to have built an innate fear of them.  This instinct to make inferences rather quickly, and to “tunnel” (i.e., focus on a small number of sources of uncertainty, or causes of known Black Swans) remains rather ingrained in us.  This instinct, in a word, is our predicament.

 

CHAPTER 6: THE NARRATIVE FALLACY

Taleb introduces the narrative fallacy:

We like stories, we like to summarize, and we like to simplify, i.e., to reduce the dimension of matters… The [narrative] fallacy is associated with our vulnerability to overinterpretation and our predilection for compact stories over raw truths.  It severely distorts our mental representation of the world; it is particularly acute when it comes to the rare event.

Taleb continues:

The narrative fallacy addresses our limited ability to look at sequences of facts without weaving an explanation into them, or, equivalently, forcing a logical link, an arrow of relationship, upon them.  Explanations bind facts together.  They make them all the more easily remembered; they help them make more sense.  Where this propensity can go wrong is when it increases our impression of understanding.

Taleb clarifies:

To help the reader locate himself: in studying the problem of induction in the previous chapter, we examined what could be inferred about the unseen, what lies outside our information set.  Here, we look at the seen, what lies within the information set, and we examine the distortions in the act of processing it.

Taleb observes that our brains automatically theorize and invent explanatory stories to explain facts.  It takes effort NOT to invent explanatory stories.

Taleb mentions post hoc rationalization.  In an experiment, women were asked to choose from among twelve pairs of nylon stockings the ones they preferred.  Then they were asked for the reasons for their choice.  The women came up with all sorts of explanations.  However, all the stockings were in fact identical.

Photo by Narokzaad

Split-brain patients have no connection between the left and right hemispheres of their brains.  Taleb:

Now, say that you induced such a person to perform an act—raise his finger, laugh, or grab a shovel—in order to ascertain how how he ascribes a reason to his act (when in fact you know that there is no reason for it other than your inducing it).  If you ask the right hemisphere, here isolated from the left side, to perform the action, then ask the other hemisphere for an explanation, the patient will invariably offer some interpretation: “I was pointing at the ceiling in order to…,” “I saw something interesting on the wall”…

Now, if you do the opposite, namely instruct the isolated left hemisphere of a right-handed person to perform an act and ask the right hemisphere for the reasons, you will be plainly told “I don’t know.”

Taleb notes that the left hemisphere deals with pattern recognition.  (But, in general, Taleb warns against the common distinctions between the left brain and the right brain.)

Taleb gives another example.  Read the following:

A BIRD IN THE

THE HAND IS WORTH

TWO IN THE BUSH

Notice anything unusual?  Try reading it again.  Taleb:

The Sydney-based brain scientist Alan Snyder… made the following discovery.  If you inhibit the left hemisphere of a right-handed person (more technically, by directing low-frequency magnetic pulses into the left frontotemporal lobes), you will lower his rate of error in reading the above caption.  Our propensity to impose meaning and concepts blocks our awareness of the details making up the concept.  However, if you zap people’s left hemispheres, they become more realistic—they can draw better and with more verisimilitude.  Their minds become better at seeing the objects themselves, cleared of theories, narratives, and prejudice.

Again, System 1 (intuition) automatically invents explanatory stories.  System 1 automatically finds patterns, even when none exist.

Moreover, neurotransmitters, chemicals thought to transport signals between different parts of the brain, play a role in the narrative fallacy.  Taleb:

It appears that pattern perception increases along with the concentration in the brain of the chemical dopamine.  Dopamine also regulates moods and supplies an internal reward system in the brain (not surprisingly, it is found in slightly higher concentrations in the left side of the brains of right-handed persons than on the right side).  A higher concentration of dopamine appears to lower skepticism and result in greater vulnerability to pattern detection;  an injection of L-dopa, a substance used to treat patients with Parkinson’s disease, seems to increase such activity and lowers one’s suspension of belief.  The person becomes vulnerable to all manner of fads…

Dopamine molecule. Illustration by Liliya623.

Our memory of the past is impacted by the narrative fallacy:

Narrativity can viciously affect the remembrance of past events as follows: we will tend to more easily remember those facts from our past that fit a narrative, while we tend to neglect others that do not appear to play a causal role in that narrative.  Consider that we recall events in our memory all the while knowing the answer of what happened subsequently.  It is literally impossible to ignore posterior information when solving a problem.  This simple inability to remember not the true sequence of events but a reconstructed one will make history appear in hindsight to be far more explainable than it actually was—or is.

Taleb again:

So we pull memories along causative lines, revising them involuntarily and unconsciously.  We continuously renarrate past events in the light of what appears to make what we think of as logical sense after these events occur.

One major problem in trying to explain and predict the facts is that the facts radically underdetermine the hypotheses that logically imply those facts.  For any given set of facts, there exist many theories that can explain and predict those facts.  Taleb:

In a famous argument, the logician W.V. Quine showed that there exist families of logically consistent interpretations and theories that can match a given set of facts.  Such insight should warn us that mere absence of nonsense may not be sufficient to make something true.

There is a way to escape the narrative fallacy.  Develop hypotheses and then run experiments that test those hypotheses.  Whichever hypotheses best explain and predict the phenomena in question can be provisionally accepted.

The best hypotheses are only provisionally true and they are never uniquely true.  The history of science shows that nearly all hypotheses, no matter how well-supported by experiments, end up being supplanted.  Odds are high that the best hypotheses of today—including general relativity and quantum mechanics—will be supplanted at some point in the future.  For example, perhaps string theory will be developed to the point where it can predict the phenomena in question with more accuracy and with more generality than both general relativity and quantum mechanics.

Taleb continues:

Let us see how narrativity affects our understanding of the Black Swan.  Narrative, as well as its associated mechanism of salience of the sensational fact, can mess up our projection of the odds.  Take the following experiment conducted by Kahneman and Tversky… : the subjects were forecasting professionals who were asked to imagine the following scenarios and estimate their odds.

Which is more likely?

    • A massive flood somewhere in America in which more than a thousand people die.
    • An earthquake in California, causing massive flooding, in which more than a thousand people die.

Most of the forecasting professionals thought that the second scenario is more likely than the first scenario.  But logically, the second scenario is a subset of the first scenario and is therefore less likely.  It’s the vividness of the second scenario that makes it appear more likely.  Again, in trying to understand these scenarios, System 1 can much more easily imagine the second scenario and so automatically views it as more likely.

Next Taleb defines two kinds of Black Swan:

…there are two varieties of rare events: a) the narrated Black Swans, those that are present in the current discourse and that you are likely to hear about on television, and b) those nobody talks about, since they escape models—those that you would feel ashamed discussing in public because they do not seem plausible.  I can safely say that it is entirely compatible with human nature that the incidences of Black Swans would be overestimated in the first case, but severely underestimated in the second one.

 

CHAPTER 7: LIVING IN THE ANTECHAMBER OF HOPE

Taleb explains:

Let us separate the world into two categories.  Some people are like the turkey, exposed to a major blowup without being aware of it, while others play reverse turkey, prepared for big events that might surprise others.  In some strategies and life situations, you gamble dollars to win a succession of pennies while appearing to be winning all the time.  In others, you risk a succession of pennies to win dollars.  In other words, you bet either that the Black Swan will happen or that it will never happen, two strategies that require completely different mind-sets.

Taleb adds:

So some matters that belong to Extremistan are extremely dangerous but do not appear to be so beforehand, since they hide and delay their risks—so suckers think they are “safe.”  It is indeed a property of Extremistan to look less risky, in the short run, than it really is.

Illustration by Mariusz Prusaczyk

Taleb describes a strategy of betting on the Black Swan:

…some business bets in which one wins big but infrequently, yet loses small but frequently, are worth making if others are suckers for them and if you have the personal and intellectual stamina.  But you need such stamina.  You also need to deal with people in your entourage heaping all manner of insult on you, much of it blatant.  People often accept that a financial strategy with a small chance of success is not necessarily a bad one as long as the success is large enough to justify it.  For a spate of psychological reasons, however, people have difficulty carrying out such a strategy, simply because it requires a combination of belief, a capacity for delayed gratification, and the willingness to be spat upon by clients without blinking.

 

CHAPTER 8: GIACOMO CASANOVA’S UNFAILING LUCK: THE PROBLEM OF SILENT EVIDENCE

Taleb:

Another fallacy in the way we understand events is that of silent evidence.  History hides both Black Swans and its Black Swan-generating ability from us.

Taleb tells the story of the drowned worshippers:

More than two thousand years ago, the Roman orator, belletrist, thinker, Stoic, manipulator-politician, and (usually) virtuous gentleman, Marcus Tullius Cicero, presented the following story.  One Diagoras, a nonbeliever in the gods, was shown painted tablets bearing the portraits of some worshippers who prayed, then survived a subsequent shipwreck.  The implication was that praying protects you from drowning.  Diagoras asked, “Where were the pictures of those who prayed, then drowned?”

This is the problem of silent evidence.  Taleb again:

As drowned worshippers do not write histories of their experiences (it is better to be alive for that), so it is with the losers in history, whether people or ideas.

Taleb continues:

The New Yorker alone rejects close to a hundred manuscripts a day, so imagine the number of geniuses that we will never hear about.  In a country like France, where more people write books while, sadly, fewer people read them, respectable literary publishers accept one in ten thousand manuscripts they receive from first-time authors.  Consider the number of actors who have never passed an audition but would have done very well had they had that lucky break in life.

Luck often plays a role in whether someone becomes a millionaire or not.  Taleb notes that many failures share the traits of the successes:

Now consider the cemetery.  The graveyard of failed persons will be full of people who shared the following traits: courage, risk taking, optimism, et cetera.  Just like the population of millionaires.  There may be some differences in skills, but what truly separates the two is for the most part a single factor: luck.  Plain luck.

Of course, there’s more luck in some professions than others.  In investment management, there’s a great deal of luck.  One way to see this is to run computer simulations.  You can see that by luck alone, if you start out with 10,000 investors, you’ll end up with a handful of investors who beat the market for 10 straight years.

(Photo by Volodymyr Pyndyk)

Taleb then gives another example of silent evidence.  He recounts reading an article about the growing threat of the Russian Mafia in the United States.  The article claimed that the toughness and brutality of these guys were because they were strengthened by their Gulag experiences.  But were they really strengthened by their Gulag experiences?

Taleb asks the reader to imagine gathering a representative sample of the rats in New York.  Imagine that Taleb subjects these rats to radiation.  Many of the rats will die.  When the experiment is over, the surviving rats will be among the strongest of the whole sample.  Does that mean that the radiation strengthened the surviving rats?  No.  The rats survived because they were stronger.  But every rat will have been weakened by the radiation.

Taleb offers another example:

Does crime pay?  Newspapers report on the criminal who get caught.  There is no section in The New York Times recording the stories of those who committed crimes but have not been caught.  So it is with cases of tax evasion, government bribes, prostitution rings, poisoning of wealthy spouses (with substances that do not have a name and cannot be detected), and drug trafficking.

In addition, our representation of the standard criminal might be based on the properties of those less intelligent ones who were caught.

Taleb next writes about politicians promising “rebuilding” New Orleans after Hurricane Katrina:

Did they promise to do so with the own money?  No.  It was with public money.  Consider that such funds will be taken away from somewhere else… That somewhere else will be less mediatized.  It may be… cancer research… Few seem to pay attention to the victims of cancer lying lonely in a state of untelevised depression.  Not only do these cancer patients not vote (they will be dead by the next ballot), but they do not manifest themselves to our emotional system.  More of them die every day than were killed by Hurricane Katrina; they are the ones who need us the most—not just our financial help, but our attention and kindness.  And they may be the ones from whom the money will be taken—indirectly, perhaps even directly.  Money (public or private) taken away from research might be responsible for killing them—in a crime that may remain silent.

Giacomo Casanova was an adventurer who seemed to be lucky.  However, there have been plenty of adventurers, so some are bound to be lucky.  Taleb:

The reader can now see why I use Casanova’s unfailing luck as a generalized framework for the analysis of history, all histories.  I generate artificial histories featuring, say, millions of Giacomo Casanovas, and observe the difference between the attributes of the successful Casanovas (because you generate them, you know their exact properties) and those an observer of the result would obtain.  From that perspective, it is not a good idea to be a Casanova.

 

CHAPTER 9: THE LUDIC FALLACY, OR THE UNCERTAINTY OF THE NERD

Taleb introduces Fat Tony (from Brooklyn):

He started as a clerk in the back office of a New York bank in the early 1980s, in the letter-of-credit department.  He pushed papers and did some grunt work.  Later he grew into giving small business loans and figured out the game of how you can get financing from the monster banks, how their bureaucracies operate, and what they like to see on paper.  All the while an employee, he started acquiring property in bankruptcy proceedings, buying it from financial institutions.  His big insight is that bank employees who sell you a house that’s not theirs just don’t care as much as the owners; Tony knew very rapidly how to talk to them and maneuver.  Later, he also learned to buy and sell gas stations with money borrowed from small neighborhood bankers.

…Tony’s motto is “Finding who the sucker is.”  Obviously, they are often the banks: “The clerks don’t care about nothing.”  Finding these suckers is second nature to him.

Next Taleb introduces non-Brooklyn John:

Dr. John is a painstaking, reasoned, and gentle fellow.  He takes his work seriously, so seriously that, unlike Tony, you can see a line in the sand between his working time and his leisure activities.  He has a PhD in electrical engineering from the University of Texas at Austin.  Since he knows both computers and statistics, he was hired by an insurance company to do computer simulations; he enjoys the business.  Much of what he does consists of running computer programs for “risk management.”

Taleb imagines asking Fat Tony and Dr. John the same question: Assume that a coin is fair.  Taleb flips the coin ninety-nine times and gets heads each time.  What are the odds that the next flip will be tails?

(Photo by Christian Delbert)

Because he assumes a fair coin and the flips are independent, Dr. John answers one half (fifty percent).  Fat Tony answers, “I’d say no more than 1 percent, of course.”  Taleb questions Fat Tony’s reasoning.  Fat Tony explains that the coin must be loaded.  In other words, it is much more likely that the coin is loaded than that Taleb got ninety-nine heads in a row flipping a fair coin.

Taleb explains:

Simply, people like Dr. John can cause Black Swans outside Mediocristan—their minds are closed.  While the problem is very general, one of its nastiest illusions is what I call the ludic fallacy—the attributes of the uncertainty we face in real life have little connection to the sterilized ones we encounter in exams and games.

Taleb was invited by the United States Defense Department to a brainstorming session on risk.  Taleb was somewhat surprised by the military people:

I came out of the meeting realizing that only military people deal with randomness with genuine, introspective intellectual honesty—unlike academics and corporate executives using other people’s money.  This does not show in war movies, where they are usually portrayed as war-hungry autocrats.  The people in front of me were not the people who initiate wars.  Indeed, for many, the successful defense policy is the one that manages to eliminate potential dangers without war, such as the strategy of bankrupting the Russians through the escalation in defense spending.  When I expressed my amazement to Laurence, another finance person who was sitting next to me, he told me that the military collected more genuine intellects and risk thinkers than most if not all other professions.  Defense people wanted to understand the epistemology of risk.

Taleb notes that the military folks had their own name for a Black Swan: unknown unknown.  Taleb came to the meeting prepared to discuss a new phrase he invented: the ludic fallacy, or the uncertainty of the nerd.

(Photo by Franky44)

In the casino you know the rules, you can calculate the odds, and the type of uncertainty we encounter there, we will see later, is mild, belonging to Mediocristan.  My prepared statement was this: “The casino is the only human venture I know where the probabilities are known, Gaussian (i.e., bell-curve), and almost computable.”…

In real life you do not know the odds; you need to discover them, and the sources of uncertainty are not defined.

Taleb adds:

What can be mathematized is usually not Gaussian, but Mandelbrotian.

What’s fascinating about the casino where the meeting was held is that the four largest losses incurred (or narrowly avoided) had nothing to do with gambling.

    • First, they lost around $100 million when an irreplaceable performer in their main show was maimed by a tiger.
    • Second, a disgruntled contractor was hurt during the construction of a hotel annex.  He was so offended by the settlement offered him that he made an attempt to dynamite the casino.
    • Third, a casino employee didn’t file required tax forms for years.  The casino ended up paying a huge fine (which was the least bad alternative).
    • Fourth, there was a spate of other dangerous scenes, such as the kidnapping of the casino owner’s daughter, which caused him, in order to secure cash for the ransom, to violate gambling laws by dipping into the casino coffers.

Taleb draws a conclusion about the casino:

A back-of-the-envelope calculation shows that the dollar value of these Black Swans, the off-model hits and potential hits I’ve just outlined, swamp the on-model risks by a factor of close to 1,000 to 1.  The casino spent hundreds of millions of dollars on gambling theory and high-tech surveillance while the bulk of their risks came from outside their models.

All this, and yet the rest of the world still learns about uncertainty and probability from gambling examples.

Taleb wraps up Part One of his book:

We love the tangible, the confirmation, the palpable, the real, the visible, the concrete, the known, the seen, the vivid, the visual, the social, the embedded, the emotionally laden, the salient, the stereotypical, the moving, the theatrical, the romanced, the cosmetic, the official, the scholarly-sounding verbiage (b******t), the pompous Gaussian economist, the mathematicized crap, the pomp, the Academie Francaise, Harvard Business School, the Nobel Prize, dark business suits with white shirts and Ferragamo ties, the moving discourse, and the lurid.  Most of all we favor the narrated.

Alas, we are not manufactured, in our current edition of the human race, to understand abstract matters—we need context.  Randomness and uncertainty are abstractions.  We respect what had happened, ignoring what could have happened.

 

PART TWO: WE JUST CAN’T PREDICT

Taleb:

…the gains in our ability to model (and predict) the world may be dwarfed by the increases in its complexity—implying a greater and greater role for the unpredicted.

 

CHAPTER 10: THE SCANDAL OF PREDICTION

Taleb highlights the story of the Sydney Opera House:

The Sydney Opera House was supposed to open in early 1963 at a cost of AU$ 7 million.  It finally opened its doors more than ten years later, and, although it was a less ambitious version than initially envisioned, it ended up costing around AU$ 104 million.

Taleb then asks:

Why on earth do we predict so much?  Worse, even, and more interesting: Why don’t we talk about our record in predicting?  Why don’t we see how we (almost) always miss the big events?  I call this the scandal of prediction.

The problem is that when our knowledge grows, our confidence about how much we know generally increases even faster.

Illustration by Airdone.

Try the following quiz.  For each question, give a range that you are 90 percent confident contains the correct answer.

    • What was Martin Luther King, Jr.’s age at death?
    • What is the length of the Nile river, in miles?
    • How many countries belong to OPEC?
    • How many books are there in the Old Testament?
    • What is the diameter of the moon, in miles?
    • What is the weight of an empty Boeing 747, in pounds?
    • In what year was Mozart born?
    • What is the gestation period of an Asian elephant, in days?
    • What is the air distance from London to Tokyo, in miles?
    • What is the deepest known point in the ocean, in feet?

If you’re not overconfident, then you should have gotten nine out of ten questions right because you gave a 90 percent confidence interval for each question.  However, most people get more than one question wrong, which means most people are overconfident.

(Answers:  39, 4132, 12, 39, 2158.8, 390000, 1756, 645, 5959, 35994.)

A similar quiz is to randomly select some number, like the population of Egypt, and then ask 100 random people to give their 98 percent confidence interval.  “I am 98 percent confident that the population of Egypt is between 40 million and 120 million.”  If the 100 random people are not overconfident, then 98 out of 100 should get the question right.  In practice, however, it turns out that a high number (15 to 30 percent) get the wrong answer.  Taleb:

This experiment has been replicated dozens of times, across populations, professions, and cultures, and just about every empirical psychologist and decision theorist has tried it on his class to show his students the big problem of humankind: we are simply not wise enough to be trusted with knowledge.  The intended 2 percent error rate usually turns out to be between 15 percent and 30 percent, depending on the population and the subject matter.

I have tested myself and, sure enough, failed, even while consciously trying to be humble by carefully setting a wide range… Yesterday afternoon, I gave a workshop in London… I decided to make a quick experiment during my talk.

I asked the participants to take a stab at a range for the number of books in Umberto Eco’s library, which, as we know from the introduction to Part One, contains 30,000 volumes.  Of the sixty attendees, not a single one made the range wide enough to include the actual number (the 2 percent error rate became 100 percent).

Taleb argues that guessing some quantity you don’t know and making a prediction about the future are logically similar.  We could ask experts who make predictions to give a confidence interval and then track over time how accurate their predictions are compared to the confidence interval.

Taleb continues:

The problem is that our ideas are sticky: once we produce a theory, we are not likely to change our minds—so those who delay developing their theories are better off.  When you develop your opinions on the basis of weak evidence, you will have difficulty interpreting subsequent information that contradicts these opinions, even if this new information is obviously more accurate.  Two mechanisms are at play here: …confirmation bias… and belief perseverance [also called consistency bias], the tendency not to reverse opinions you already have.  Remember that we treat ideas like possessions, and it will be hard for us to part with them.

…the more detailed knowledge one gets of empirical reality, the more one will see the noise (i.e., the anecdote) and mistake it for actual information.  Remember that we are swayed by the sensational.

Taleb adds:

…in another telling experiment, the psychologist Paul Slovic asked bookmakers to select from eighty-eight variables in past horse races those that they found useful in computing the odds.  These variables included all manner of statistical information about past performances.  The bookmakers were given the ten most useful variables, then asked to predict the outcome of races.  Then they were given ten more and asked to predict again.  The increase in the information set did not lead to an increase in their accuracy; their confidence in their choices, on the other hand, went up markedly.  Information proved to be toxic.

When it comes to dealing with experts, many experts do have a great deal of knowledge.  However, most experts have a high error rate when it comes to making predictions.  Moreover, many experts don’t even keep track of how accurate their predictions are.

Another way to think about the problem is to try to distinguish those with true expertise from those without it.  Taleb:

    • Experts who tend to be experts: livestock judges, astronomers, test pilots, soil judges, chess masters, physicists, mathematicians (when they deal with mathematical problems, not empirical ones), accountants, grain inspectors, photo interpreters, insurance analysts (dealing with bell curve-style statistics).
    • Experts who tend to be… not experts: stockbrokers, clinical psychologists, psychiatrists, college admissions officers, court judges, councilors, personnel selectors, intelligence analysts… economists, financial forecasters, finance professors, political scientists, “risk experts,” Bank for International Settlements staff, august members of the International Association of Financial Engineers, and personal financial advisors.

Taleb comments:

You cannot ignore self-delusion.  The problem with experts is that they do not know what they do not know.  Lack of knowledge and delusion about the quality of you knowledge come together—the same process that makes you know less also makes you satisfied with your knowledge.

Taleb asserts:

Our predictors may be good at predicting the ordinary, but not the irregular, and this is where they ultimately fail.  All you need to do is miss one interest-rates move, from 6 percent to 1 percent in a longer-term projection (what happened between 2000 and 2001) to have all your subsequent forecasts rendered completely ineffectual in correcting your cumulative track record.  What matters is not how often you are right, but how large your cumulative errors are.

And these cumulative errors depend largely on the big surprises, the big opportunities.  Not only do economic, financial, and political predictors miss them, but they are quite ashamed to say anything outlandish to their clients—and yet events, it turns out, are almost always outlandish.  Furthermore… economic forecasters tend to fall closer to one another than to the resulting outcome.  Nobody wants to be off the wall.

Taleb notes a paper that analyzed two thousand predictions by brokerage-house analysts.  These predictions didn’t predict anything at all.  You could have done about as well by naively extrapolating the prior period to the next period.  Also, the average difference between the forecasts was smaller than the average error of the forecasts.  This indicates herding.

Taleb then writes about the psychologist Philip Tetlock’s research.  Tetlock analyzed twenty-seven thousand predictions by close to three hundred specialists.  The predictions took the form of more of x, no change in x, or less of x.  Tetlock found that, on the whole, these predictions by experts were little better than chance.  You could have done as well by rolling a dice.

Tetlock worked to discover why most expert predictors did not realize that they weren’t good at making predictions.  He came up with several methods of belief defense:

    • You tell yourself that you were playing a different game.  Virtually no social scientist predicted the fall of the Soviet Union.  You argue that the Russians had hidden the relevant information.  If you’d had enough information, you could have predicted the fall of the Soviet Union.  “It is not your skills that are to blame.”
    • You invoke the outlier.  Something happened that was outside the system.  It was a Black Swan, and you are not supposed to predict Black Swans.  Such events are “exogenous,” coming from outside your science.  The model was right, it worked well, but the game turned out to be a different one than anticipated.
    • The “almost right” defense.  Retrospectively, it is easy to feel that it was a close call.

Taleb writes:

We attribute our successes to our skills, and our failures to external events outside our control, namely to randomness… This causes us to think that we are better than others at whatever we do for a living.  Nine-four percent of Swedes believe that their driving skills  put them in the top 50 percent of Swedish drivers; 84 percent of Frenchmen feel that their lovemaking abilities put them in the top half of French lovers.

Taleb observes that we tend to feel a little unique, unlike others.  If we get married, we don’t consider divorce a possibility.  If we buy a house, we don’t think we’ll move.  People who lose their job often don’t expect it.  People who try drugs don’t think they’ll keep doing it for long.

Taleb says:

Tetlock distinguishes between two types of predictors, the hedgehog and the fox, according to a distinction promoted by the essayist Isaiah Berlin.  As in Aesop’s fable, the hedgehog knows one thing, the fox knows many things… Many of the prediction failures come from hedgehogs who are mentally married to a single big Black Swan event, a big bet that is not likely to play out.  The hedgehog is someone focusing on a single, improbable, and consequential event, falling for the narrative fallacy that makes us so blinded by one single outcome that we cannot imagine others.

Taleb makes it clear that he thinks we should be foxes, not hedgehogs.  Taleb has never tried to predict specific Black Swans.  Rather, he wants to be prepared for whatever might come.  That’s why it’s better to be a fox than a hedgehog.  Hedgehogs are much worse, on the whole, at making predictions than foxes are.

Taleb mentions a study by Spyros Makridakis and Michele Hibon of predictions made using econometrics.  They discovered that “statistically sophisticated or complex methods” are not clearly better than simpler ones.

Projects usually take longer and are more expensive than most people think.  For instance, students regularly underestimate how long it will take them to complete a class project.  Taleb then adds:

With projects of great novelty, such as a military invasion, an all-out war, or something entirely new, errors explode upward.  In fact, the more routine the task, the better you learn to forecast.  But there is always something nonroutine in our modern environment.

Taleb continues:

…we are too focused on matters internal to the project to take into account external uncertainty, the “unknown unknown,” so to speak, the contents of the unread books.

Another important bias to understand is anchoring.  The human brain, relying on System 1, will grab on to any number, no matter how random, as a basis for guessing some other quantity.  For example, Kahneman and Tversky spun a wheel of fortune in front of some people.  What the people didn’t know was that the wheel was pre-programmed to either stop at “10” or “65.”  After the wheel stopped, people were asked to write down their guess of the number of African countries in the United Nations.  Predictably, those who saw “10” guessed a much lower number (25% was the average guess) than those who saw “65” (45% was the average guess).

Next, Taleb points out that life expectancy is from Mediocristan.  It is not scalable.  The longer we live, the less long we are expected to live.  By contrast, projects and ventures tend to be scalable.  The longer we have waited for some project to be completed, the longer we can be expected to have to wait from that point forward.

Taleb gives the example of a refugee waiting to return to his or her homeland.  The longer the refugee has waited so far, the longer they should expect to have to wait going forward.  Furthermore, consider wars: they tend to last longer and kill more people than expected.  The average war may last six months, but if your war has been going on for a few years, expect at least a few more years.

Taleb argues that corporate and government projections have an obvious flaw: they do not include an error rate.  There are three fallacies involved:

    • The first fallacy: variability matters.  For planning purposes, the accuracy of your forecast matters much more than the forecast itself, observes Taleb.  Don’t cross a river if it is four feet deep on average.  Taleb gives another example.  If you’re going on a trip to a remote location, then you’ll pack different clothes if it’s supposed to be seventy degrees Fahrenheit with an expected error rate of forty degrees than if the margin of error was only five degrees.  “The policies we need to make decisions on should depend far more on the range of possible outcomes than on the expected final number.”
    • The second fallacy lies in failing to take into account forecast degradation as the projected period lengthens… Our forecast errors have traditionally been enormous…
    • The third fallacy, and perhaps the gravest, concerns a misunderstanding of the random character of the variables being forecast.  Owing to the Black Swan, these variables can accomodate far more optimistic—or far more pessimistic—scenarios than are currently expected.

Taleb points out that, as in the case of the depth of the river, what matters even more than the error rate is the worst-case scenario.

A Black Swan has three attributes: unpredictability, consequences, and retrospective explainability.  Taleb next examines unpredictability.

 

CHAPTER 11: HOW TO LOOK FOR BIRD POOP

Taleb notes that most discoveries are the product of serendipity.

Photo by Marek Uliasz

Taleb writes:

Take this dramatic example of a serendipitous discovery.  Alexander Fleming was cleaning up his laboratory when he found that penicillium mold had contaminated one of his old experiments.  He thus happened upon the antibacterial properties of penicillin, the reason many of us are alive today (including…myself, for typhoid fever is often fatal when untreated)… Furthermore, while in hindsight the discovery appears momentous, it took a very long time for health officials to realize the importance of what they had on their hands.  Even Fleming lost faith in the idea before it was subsequently revived.

In 1965 two radio astronomists at Bell Labs in New Jersey who were mounting a large antenna were bothered by a background noise, a hiss, like the static that you hear when you have bad reception.  The noise could not be eradicated—even after they cleaned the bird excrement out of the dish, since they were convinced that bird poop was behind the noise.  It took a while for them to figure out that what they were hearing was the trace of the birth of the universe, the cosmic background microwave radiation.  This discovery revived the big bang theory, a languishing idea that was posited by earlier researchers.

What’s interesting (but typical) is that the physicists—Ralph Alpher, Hans Bethe, and George Gamow—who conceived of the idea of cosmic background radiation did not discover the evidence they were looking for, while those not looking for such evidence found it.

Furthermore, observes Taleb:

When a new technology emerges, we either grossly underestimate or severely overestimate its importance.  Thomas Watson, the founder of IBM, once predicted that there would be no need for more than just a handful of computers.

Taleb adds:

The laser is a prime illustration of a tool made for a given  purpose (actually no real purpose) that then found applications that were not even dreamed of at the time.  It was a typical “solution looking for a problem.”  Among the early applications was the surgical stitching of detached retinas.  Half a century later, The Economist asked Charles Townes, the alleged inventor of the laser, if he had had retinas on his mind.  He had not.  He was satisfying his desire to split light beams, and that was that.  In fact, Townes’s colleagues teased him quite a bit about the irrelevance of his discovery.  Yet just consider the effects of the laser in the world around you: compact disks, eyesight corrections, microsurgery, data storage and retrieval—all unforeseen applications of the technology.

Taleb mentions that the French mathematician Henri Poincare was aware that equations have limitations when it comes to predicting the future.

Poincare’s reasoning was simple: as you project into the future you may need an increasing amount of precision about the dynamics of the process that you are modeling, since your error rate grows very rapidly… Poincare showed this in a very simple case, famously known as the “three body problem.”  If you have only two planets in a solar-style system, with nothing else affecting their course, then you may be able to indefinitely predict the behavior of these planets, no sweat.  But add a third body, say a comet, ever so small, between the planets.  Initially the third body will cause no drift, no impact; later, with time, its effects on the other two bodies may become explosive.

Our world contains far more than just three bodies.  Therefore, many future phenomena are unpredictable due to complexity.

The mathematician Michael Berry gives another example: billiard balls.  Taleb:

If you know a set of basic parameters concerning the ball at rest, can compute the resistance of the table (quite elementary), and can gauge the strength of the impact, then it is rather easy to predict what would happen at the first hit… The problem is that to correctly predict the ninth impact, you need to take into account the gravitational pull of someone standing next to the table… And to compute the fifty-sixth impact, every single elementary particle of the universe needs to be present in your assumptions!

Moreover, Taleb points out, in the billiard ball example, we don’t have to worry about free will.  Nor have we incorporated relativity and quantum effects.

You can think rigorously, but you cannot use numbers.  Poincare even invented a field for this, analysis in situ, now part of topology…

In the 1960s the MIT meteorologist Edward Lorenz rediscovered Poincare’s results on his own—once again, by accident.  He was producing a computer model of weather dynamics, and he ran a simulation that projected a weather system a few days ahead.  Later he tried to repeat the same simulation with the exact same model and what he thought were the same input parameters, but he got wildly different results… Lorenz subsequently realized that the consequential divergence in his results arose not from error, but from a small rounding in the input parameters.  This became known as the butterfly effect, since a butterfly moving its wings in India could cause a hurricane in New York, two years later.  Lorenz’s findings generated interest in the field of chaos theory.

Much economics has been developed assuming that agents are rational.  However, Kahneman and Tversky have shown—in their work on heuristics and biases—that many people are less than fully rational.  Kahneman and Tversky’s experiments have been repeated countless times over decades.  Some people prefer apples to oranges, oranges to pears, and pears to apples.  These people do not have consistent preferences.  Furthermore, when guessing at an unknown quantity, many people will anchor on any random number even though the random number often has no relation to the quantity guessed at.

People also make different choices based on framing effects.  Kahneman and Tversky have illustrated this with the following experiment in which 600 people are assumed to have a deadly disease.

First Kahneman and Tversky used a positive framing:

    • Treatment A will save 200 lives for sure.
    • Treatment B has a 33% chance of saving everyone and a 67% chance of saving no one.

With this framing, 72% prefer Treatment A and 28% prefer Treatment B.

Next a negative framing:

    • Treatment A will kill 400 people for sure.
    • Treatment B has a 33% chance of killing no one and a 67% chance of killing everyone.

With this framing, only 22% prefer Treatment A, while 78% prefer Treatment B.

Note:  The two frames are logically identical, but the first frame focuses on lives saved, whereas the second frame focuses on lives lost.

Taleb argues that the same past data can confirm a theory and also its exact opposite.  Assume a linear series of points.  For the turkey, that can either confirm safety or it can mean the turkey is much closer to being turned into dinner.  Similarly, as Taleb notes, each day you live can either mean that you’re more likely to be immortal or that you’re closer to death.  Taleb observes that a linear regression model can be enormously misleading if you’re in Extremistan: Just because the data thus far appear to be in a straight line tells you nothing about what’s to come.

Taleb says the philosopher Nelson Goodman calls this the riddle of induction:

Let’s say that you observe an emerald.  It was green yesterday and the day before yesterday.  It is green again today.  Normally this would confirm the “green” property: we can assume that the emerald will be green tomorrow.  But to Goodman, the emerald’s color history could equally confirm the “grue” property.  What is this grue property?  The emerald’s grue property is to be green until some specified date… and then blue thereafter.

The riddle of induction is another version of the narrative fallacy—you face an infinity of “stories” that explain what you have seen.  The severity of Goodman’s riddle of induction is as follows: if there is no longer even a single unique way to “generalize” from what you see, to make an inference about the unknown, then how should you operate?  The answer, clearly, will be that you should employ “common sense,” but your common sense may not be so well developed with respect to some Extremistan variables.

 

CHAPTER 12: EPISTEMOCRACY, A DREAM

Taleb defines an epistemocrat as someone who is keenly aware that his knowledge is suspect.  Epistemocracy is a place where the laws are made with human fallibility in mind.  Taleb says that the major modern epistemocrat is the French philosopher Michel de Montaigne.

Montaigne is quite refreshing to read after the strains of a modern education since he fully accepted human weaknesses and understood that no philosophy could be effective unless it took into account our deeply ingrained imperfections, the limitations of our rationality, the flaws that make us human.  It is not that he was ahead of his time; it would be better said that later scholars (advocating rationality) were backward.

Photo by Jacek Dudzinski

Montaigne was not just a thinker, but also a doer.  He had been a magistrate, a businessman, and the mayor of Bordeaux.  Taleb writes that Montaigne was a skeptic, an antidogmatist.

So what would an epistemocracy look like?

The Black Swan asymmetry allows you to be confident about what is wrong, not about what you believe is right.

Taleb adds:

The notion of future mixed with chance, not a deterministic extension of your perception of the past, is a mental operation that our mind cannot perform.  Chance is too fuzzy for us to be a category by itself.  There is an asymmetry between past and future, and it is too subtle for us to understanding naturally.

The first consequence of this asymmetry is that, in people’s minds, the relationship between the past and the future does not learn from the relationship between the past and the past previous to it.  There is a blind spot: when we think of tomorrow we do not frame it in terms of what we thought about yesterday or the day before yesterday.  Because of this introspective defect we fail to learn about the difference between our past predictions and the subsequent outcomes.  When we think of tomorrow, we just project it as another yesterday.

As yet another example of how we can’t predict, psychologists have shown that we can’t predict our future affective states in response to both pleasant and unpleasant events.  The point is that we don’t learn from our past errors in predicting our future affective states.  We continue to make the same mistake by overestimating the future impact of both pleasant and unpleasant events.  We persist in thinking that unpleasant events will make us more unhappy than they actually do.  We persist in thinking that pleasant events will make us happier than they actually do.  We simply don’t learn from the fact that we made these erroneous predictions in the past.

Next Taleb observes that it’s not only that we can’t predict the future.  We don’t know the past either.  Taleb gives this example:

    • Operation 1 (the melting ice cube): Imagine an ice cube and consider how it may melt over the next two hours while you play a few rounds of poker with your friends.  Try to envision the shape of the resulting puddle.
    • Operation 2 (where did the water come from?): Consider a puddle of water on the floor.  Now try to reconstruct in your mind’s eye the shape of the ice cube it may once have been.  Note that the puddle may not have necessarily originated from an ice cube.

It’s one thing to use physics and engineering to predict the forward process of an ice cube melting.  It’s quite another thing to try to reconstruct what it was that led to a puddle of water.  Taleb:

In a way, the limitations that prevent us from unfrying an egg also prevent us from reverse engineering history.

For these reasons, history should just be a collection of stories, argues Taleb.  History generally should not try to discover the causes of why things happened the way they did.

 

CHAPTER 13: APELLES THE PAINTER, OR WHAT DO YOU DO IF YOU CANNOT PREDICT?

Taleb writes about being a fool in the right places:

The lesson for the small is: be human!  Accept that being human involves some amount of epistemic arrogance in running your affairs.  Do not be ashamed of that.  Do not try to always withhold judgment—opinions are the stuff of life.  Do not try to avoid predicting—yes, after this diatribe about prediction I am not urging you to stop being a fool.  Just be a fool in the right places.

What you should avoid is unnecessary dependence on large-scale harmful predictions—those and only those.  Avoid the big subjects that may hurt your future: be fooled in small matters, not in the large.  Do not listen to economic forecasters or to predictors in social science (they are mere entertainers), but do make your own forecast for the picnic…

Know how to rank beliefs not according to their plausibility but by the harm they may cause.

Taleb advises us to maximize the serendipity around us.  You want to be exposed to the positive accident.  Taleb writes that Sextus Empiricus retold a story about Apelles the Painter.  Try as he might, Apelles was not able to paint the foam on a horse’s mouth.  He really tried hard but eventually gave up.  In irritation, he took a sponge and threw it at the painting.  The sponge left a pattern on the painting that perfectly depicted the foam.

(Illustration by Ileezhun)

Taleb recommends trial and error:

Indeed, we have psychological and intellectual difficulties with trial and error, and with accepting that series of small failures are necessary in life.  My colleague Mark Spitznagel understood that we humans have a mental hang-up about failures:  “You need to love to lose” was his motto.  In fact, the reason I felt immediately at home in America is precisely because American culture encourages the process of failure, unlike the cultures of Europe and Asia where failure is met with stigma and embarrassment.  America’s specialty is to take these small risks for the rest of the world, which explains this country’s disproportionate share in innovations.

Taleb then points out:

People are often ashamed of losses, so they engage in strategies that produce very little volatility but contain the risk of a large loss—like collecting nickles in front of steamrollers.

Taleb recommends a barbell strategy.  You put 85 to 90 percent of your money in extremely safe instruments like U.S. Treasury bills.

The remaining 10 to 15 percent you put in extremely speculative bets, as leveraged as possible (like options), preferably venture capital-style portfolios.

Taleb offers five tricks:

    • Make a distinction between positive contingencies and negative ones.  There are both positive and negative Black Swans.
    • Do not be narrow-minded.  Do not try to predict precise Black Swans because that tends to make you more vulnerable to the ones you didn’t predict.  Invest in preparedness, not in prediction.
    • Seize any opportunity, or anything that looks like opportunity.  They are rare, much rarer than you think.  Work hard, not in grunt work, but in chasing such opportunities and maximizing exposure to them.
    • Beware of precise plans by governments.
    • Do not waste your time trying to fight forecasters, stock analysts, economists, and social scientists.  People will continue to predict foolishly, especially if they are paid for it.

Taleb concludes:

All these recommendations have one point in common: asymmetry.  Put yourself in situations where favorable consequences are much larger than unfavorable ones.

Taleb explains:

This idea that in order to make a decision you need to focus on the consequences (which you can know) rather than the probability (which you can’t know) is the central idea of uncertainty.  Much of my life is based on it.

 

PART THREE: THOSE GRAY SWANS OF EXTREMISTAN

The final four items related to the Black Swan:

    • The world is moving deeper into Extremistan.
    • The Gaussian bell curve is a contagious and severe delusion.
    • Using Mandelbrotian, or fractal, randomness, some Black Swans can by turned into Gray Swans.
    • Some ideas about uncertainty.

 

CHAPTER 14: FROM MEDIOCRISTAN TO EXTREMISTAN, AND BACK

The economist Sherwin Rosen wrote in the early 1980s about “the economics of superstars.”  Think about some of the best professional athletes or actors earning hundreds of millions of dollars.

According to Rosen, this inequality comes from a tournament effect: someone who is marginally “better” can easily win the entire pot, leaving the others with nothing…

But the role of luck is missing in Rosen’s beautiful argument.  The problem here is the notion of “better,” this focus on skills as leading to success.  Random outcomes, or an arbitrary situation, can also explain success, and provide the initial push that leads to a winner-take-all result.  A person can get slightly ahead for entirely random reasons; because we like to imitate one another, we will flock to him.  The world of contagion is so underestimated!

As I am writing these lines, I am using a Macintosh, by Apple, after years of using Microsoft-based products.  The Apple technology is vastly better, yet the inferior software won the day.  How?  Luck.

Taleb next mentions the Matthew effect, according to which people take from the poor to give to the rich.  Robert K. Merton looked at the performance of scientists and found that an initial advantage would tend to follow someone through life.  The theory also can apply to companies, businessmen, actors, writers, and anyone else who benefits from past success.

Taleb observes that the vast majority of the largest five hundred U.S. corporations have eventually either shrunk significantly or gone out of business.  Why?  Luck plays a large role.

Photo by Pat Lalli

Taleb:

Capitalism is, among other things, the revitalization of the world thanks to the opportunity to be lucky.  Luck is the grand equalizer, because almost everyone can benefit from it…

Everything is transitory.  Luck both made and unmade Carthage; it both made and unmade Rome.

I said earlier that randomness is bad, but it is not always so.  Luck is far more egalitarian than even intelligence.  If people were rewarded strictly according to their abilities, things would still be unfair—people don’t choose their abilities.  Randomness has the beneficial effect of reshuffling society’s cards, knocking down the big guy.

 

CHAPTER 15: THE BELL CURVE, THAT GREAT INTELLECTUAL FRAUD

The Gaussian bell curve, also called the normal distribution, describes many things, including height.  Taleb presents the following data about height.  First, he assumes that the average height (men and women) is 1.67 meters, (about 5 feet 7 inches).  Then look at the following increments and consider the odds of someone being that tall:

  • 10 centimeters taller than the average (1.77 m, or 5 feet 10): 1 in 6.3
  • 20 centimeters taller than the average (1.87 m, or 6 feet 2): 1 in 44
  • 30 centimeters taller than the average (1.97 m, or 6 feet 6): 1 in 740
  • 40 centimeters taller than the average (2.07 m, or 6 feet 9): 1 in 32,000
  • 50 centimeters taller than the average (2.17 m, or 7 feet 1): 1 in 3,500,000
  • 60 centimeters taller than the average (2.27 m, or 7 feet 5): 1 in 1,000,000,000
  • 70 centimeters taller than the average (2.37 m, or 7 feet 9): 1 in 780,000,000,000
  • 80 centimeters taller than the average (2.47 m, or 8 feet 1): 1 in 1,600,000,000,000,000
  • 90 centimeters taller than the average (2.57 m, or 8 feet 5): 1 in 8,900,000,000,000,000,000
  • 100 centimeters taller than the average (2.67 m, or 8 feet 9): 1 in 130,000,000,000,000,000,000,000
  • 110 centimeters taller than the average (2.77 m, or 9 feet 1): 1 in 36,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000,000

Taleb comments that the super fast decline is what allows you to ignore outliers in a normal distribution (the bell curve).

By contrast, consider the odds of being rich in Europe:

    • People with a net worth higher than 1 million: 1 in 63
    • Higher than 2 million: 1 in 250
    • Higher than 4 million: 1 in 1,000
    • Higher than 8 million: 1 in 4,000
    • Higher than 16 million: 1 in 16,000
    • Higher than 32 million: 1 in 64,000
    • Higher than 320 million: 1 in 6,400,000

The important point is that for this Mandelbrotian distribution, the speed of the decrease remains constant.

(Power law graph, via Wikimedia Commons)

Taleb:

This, in a nutshell, illustrates the difference between Mediocristan and Extremistan.

Taleb writes:

Consider this effect.  Take a random sample of any two people from the U.S. population who jointly earn $1 million per annum.  What is the most likely breakdown of their respective incomes?  In Mediocristan, the most likely combination is half a million each.  In Extremistan, it would be $50,000 and $950,000.

The situation is even more lopsided with book sales.  If I told you that two authors sold a total of a million copies of their books, the most likely combination is 993,000 copies sold for one and 7,000 for the other.  This is far more likely than that the books each sold 500,000 copies…

Why is this so?  The height problem provides a comparison.  If I told you that the total height of two people is fourteen feet, you would identify the most likely breakdown as seven feet each, not two feet and twelve feet; not even eight feet and six feet!

Taleb summarizes:

Although unpredictable large deviations are rare, they cannot be dismissed as outliers because, cumulatively, their impact is so dramatic.

The traditional Gaussian way of looking at the world begins by focusing on the ordinary, and then deals with exceptions or so-called outliers as ancillaries.  But there is a second way, which takes the exceptional as a starting point and treats the ordinary as subordinate.

Taleb continues by noting that if there are strong forces bringing things back into equilibrium, then you can use the Gaussian approach.  (That’s why much of economics is based on equilibrium.)  If there is a rational reason for the largest not to be too far from the average, then again you can use the Gaussian approach.  If there are physical limitations preventing very large observations, once again the Gaussian approach works.

Another example of where the Gaussian approach works is a cup of coffee.  There are several trillion particles in a coffee cup.  But for the cup of coffee to jump off a table, all the particles would have to jump in the same direction.  That’s not going to happen in the lifetime of this universe, notes Taleb.

Taleb explains that the Gaussian family is the only class of distributions for which the average and the standard deviation are sufficient to describe.  Moreover, correlation and regression have little or no significance outside of the Gaussian.  Taleb observes that correlation and standard deviation can be very unstable and can depend largely on which historical periods you look at.

The French mathematician Poincare was suspicious of the Gaussian, writes Taleb.

Poincare wrote that one of his friends, an unnamed “eminent physicist,” complained to him that physicists tended to use the Gaussian curve because they thought mathematicians believed it a mathematical necessity; mathematicians used it because they believed that physicists found it to be an empirical fact.

Taleb adds:

If you’re dealing with qualitative inference, such as in psychology or medicine, looking for yes/no answers to which magnitudes don’t apply, then you can assume you’re in Mediocristan without serious problems.  The impact of the improbable cannot be too large.  You have cancer or you don’t, you are pregnant or you are not, et cetera… But if you are dealing with aggregates, where magnitudes do matter, such as income, your wealth, return on a portfolio, or book sales, then you will have a problem and get the wrong distribution if you use the Gaussian, as it does not belong there.  One single number can disrupt all your averages; one single loss can eradicate a century of profits.

 

CHAPTER 16: THE AESTHETICS OF RANDOMNESS

Taleb:

Fractality is the repetition of geometric patterns at different scales, revealing smaller and smaller versions of themselves.

Taleb explains:

There is no qualitative change when an object changes size.  If you look at the coast of Britain from an airplane, it resembles what you see when you look at it with a magnifying glass.  This character of self-affinity implies that one deceptively short and simple rule of iteration can be used, either by a computer or, more randomly, by Mother Nature, to build shapes of seemingly great complexity… Mandelbrot designed the mathematical object now known as the Mandelbrot set, the most famous object in the history of mathematics.  It became popular with followers of chaos theory because it generates pictures of ever increasing complexity by using a deceptively minuscule recursive rule; recursive means that something can be reapplied to itself infinitely.  You can look at the set at smaller and smaller resolutions without ever reaching the limit; you will continue to see recognizable shapes.  The shapes are never the same, yet they bear an affinity to one another, a strong family resemblance.

Taleb notes that most computer-generated objects are based on some version of the Mandelbrotian fractal.  Taleb writes of Benoit Mandelbrot:

His talks were invaded by all sorts of artists, earning him the nickname the Rock Star of Mathematics.  The computer age helped him become one of the most influential mathematicians in history, in terms of the applications of his work, way before his acceptance by the ivory tower.  We will see that, in addition to its universality, his work offers an unusual attribute: it is remarkably easy to understand.

Let’s consider again Mediocristan.  Taleb:

I am looking at the rug in my study.  If I examine it with a microscope, I will see a very rugged terrain.  If I look at it with a magnifying glass, the terrain will be smoother but still highly uneven.  But when I look at it from a standing position, it appears uniform—it is almost as smooth as a sheet of paper.  The rug at eye level corresponds to Mediocristan and the law of large numbers: I am seeing the sum of undulations, and these iron out.  This is like Gaussian randomness: the reason my cup of coffee does not jump is that the sum of all of its moving particles becomes smooth.  Likewise, you reach certainties by adding up small Gaussian uncertainties: this is the law of large numbers.

The Gaussian is not self-similar, and that is why my coffee cup does not jump on my desk.

How does fractal geometry relate to things like the distribution of wealth, the size of cities, returns in the financial markets, the number of casualties in war, or the size of planets?  Taleb:

The key here is that the fractal has numerical or statistical measures that are (somewhat) preserved across scales—the ratio is the same, unlike the Guassian.

Taleb argues that fractals can make Black Swans gray:

Fractals should be the default, the approximation, the framework.  They do not solve the Black Swan problem and do not turn all Black Swans into predictable events, but they significantly mitigate the Black Swan problem by making such large events conceivable.

Taleb continues:

I have shown in the wealth lists in Chapter 15 the logic of a fractal distribution: if wealth doubles from 1 million to 2 million, the incidence of people with at least that much money is cut in four, which is an exponent of two.  If the exponent were one, then the incidence of that wealth or more would be cut in two.  The exponent is called the “power” (which is why some people use the term power law).

Taleb presents the following table with the assumed exponents (powers) for various phenomena:

Phenomenon Assumed Exponent (vague approximation)
Frequency of use of words 1.2
Number of hits on websites 1.4
Number of books sold in the U.S. 1.5
Telephone calls received 1.22
Magnitude of earthquakes 2.8
Diameter of moon craters 2.14
Intensity of solar flares 0.8
Intensity of wars 0.8
Net worth of Americans 1.1
Number of persons per family name 1
Population of U.S. cities 1.3
Market moves 3 (or lower)
Company size 1.5
People killed in terrorist attacks 2 (but possibly a much lower exponent)

Note that these exponents (powers) are best guesses on the basis of statistical information.  It’s often hard to know the true parametersif they exist.  Also note that you will have a huge sampling error.  Finally, note that, because of the way the math works (you use the negative of the exponent), a lower power implies greater deviations.

Taleb observes:

My colleagues and I worked with around 20 million pieces of financial data.  We all had the same data set, yet we never agreed on exactly what the exponent was in our sets.  We knew the data revealed a fractal power law, but we learned that one could not produce a precise number.  But what we did knowthat the distribution is scalable and fractalwas sufficient for us to operate and make decisions.

 

CHAPTER 17: LOCKE’S MADMEN, OR BELL CURVES IN THE WRONG PLACES

Taleb laments the fact that Gaussian tools are still widely used even when they don’t apply to the phenomena in question:

The strangest thing is that people in business usually agree with me when they listen to me talk or hear me make my case.  But when they go to the office the next day they revert to the Gaussian tools so entrenched in their habits.  Their minds are domain-dependent, so they can exercise critical thinking at a conference while not doing so in the office.  Furthermore, the Gaussian tools give them numbers, which seem to be “better than nothing.”  The resulting measure of future uncertainty satisfies our ingrained desire to simplify even if that means squeezing into one single number matters that are too rich to be described that way.

Taleb later describes how various researchers disputed Taleb’s main points:

People would find data in which there were no jumps or extreme events, and show me a “proof” that one could use the Gaussian.  [This is like observing O.J. Simpson and concluding he’s not a killer because you never saw him kill someone while you were observing him.]  The entire statistical business confused absence of proof with proof of absence.  Furthermore, people did not understand the elementary asymmetry involved: you need one single observation to reject the Gaussian, but millions of observations will not fully confirm the validity of its application.  Why?  Because the Gaussian bell curve disallows large deviations, but tools of Extremistan, the alternative, do not disallow long quiet stretches.

The hedge fund Long-Term Capital Management, or LTCM, was founded by people considered to be geniuses, including Nobel winners Robert Merton, Jr., and Myron Scholes.  LTCM, using Gaussian methods, claimed that it had very sophisticated ways of measuring risk.  According to their Gaussian models, they had virtually no real risk.  Then LTCM blew up.  A Black Swan.

Taleb comments:

The magnitude of the losses was spectacular, too spectacular to allow us to ignore the intellectual comedy.  Many friends and I though that the portfolio theorists would suffer the fate of tobacco companies: they were endangering people’s savings and would soon be brought to account for the consequences of their Gaussian-inspired methods.

None of that happened.

Instead, MBAs in business schools went on learning portfolio theory.  And the option formula went on bearing the name Black-Scholes-Merton, instead of reverting to its true owners, Louis Bachelier, Ed Thorp, and others.

Despite the overwhelming evidence that Gaussian assumptions do not apply to past financial data, many researchers continue to make Gaussian assumptions.  Taleb says this resembles Locke’s definition of a madman: someone “reasoning correctly from erroneous premises.”

Taleb asserts that military people focus first on having realistic assumptions.  Only later do they focus on correct reasoning.

This is where you learn from the minds of military people and those who have responsibilities in security.  They do not care about “perfect” ludic reasoning; they want realistic ecological assumptions.  In the end, they care about lives.

 

CHAPTER 18: THE UNCERTAINTY OF THE PHONY

People like to refer to the uncertainty principle and then talk about the limits of our knowledge.  However, uncertainties about subatomic particles are very small and very numerous.  They average out, says Taleb: They obey the law of large numbers and they are Gaussian.

Taleb writes about trying to visit his ancestral village of Amioun, Lebanon:

Beirut’s airport is closed owing to the conflict between Israel and the Shiite militia Hezbollah.  There is no published airline schedule that will inform me when the war will end, if it ends.  I can’t figure out if my house will be standing, if Amioun will still be on the maprecall that the family house was destroyed once before.  I can’t figure out if the war is going to degenerate into something even more severe.  Looking into the outcome of the war, with all my relatives, friends, and property exposed to it, I face true limits of knowledge.  Can someone explain to me why I should care about subatomic particles that, anyway, converge to a Gaussian?  People can’t predict how long they will be happy with recently acquired objects, how long their marriages will last, how their new jobs will turn out, yet it’s subatomic particles that they cite as “limits of prediction.”  They’re ignoring a mammoth standing in front of them in favor of matter even a microscope would not allow them to see.

 

PART FOUR: THE END

CHAPTER 19: HALF AND HALF, OR HOW TO GET EVEN WITH THE BLACK SWAN

Taleb concludes:

Half the time I am a hyperskpetic; the other half I hold certainties and can be intransigent about them, with a very stubborn disposition… I am skeptical when I suspect wild randomness, gullible when I believe that randomness is mild.

Half the time I hate Black Swans, the other half I love them.  I like the randomness that produces the texture of life, the positive accidents, the success of Apelles the painter, the potential gifts you do not have to pay for.  Few understand the beauty in the story of Apelles; in fact, most people exercise their error avoidance by repressing the Apelles in them.

Taleb continues:

In the end this is a trivial decision making rule: I am very aggressive when I can gain exposure to positive Black Swanswhen a failure would be of small momentand very conservative when I am under threat from a negative Black Swan.  I am very aggressive when an error in a model can benefit me, and paranoid when the error can hurt.  This may not be too interesting except that it is exactly what other people do not do…

Half the time I am intellectual, the other half I am a no-nonsense practitioner.  I am no-nonsense and practical in academic matters, and intellectual when it comes to practice.

Half the time I am shallow, the other half I want to avoid shallowness.  I am shallow when it comes to aesthetics; I avoid shallowness in the context of risks and returns.  My aestheticism makes me put poetry before prose, Greeks before Romans, dignity before elegance, elegance before culture, culture before erudition, erudition before knowledge, knowledge before intellect, and intellect before truth.  But only for matters that are Black Swan free.  Our tendency is to be very rational, except when it comes to the Black Swan.

Taleb’s final points:

We are quick to forget that just being alive is an extraordinary piece of good luck, a remote event, a chance occurrence of monstrous proportions.

Imagine a speck of dust next to a planet a billion times the size of the earth.  The speck of dust represents the odds in favor of your being born; the huge planet would be the odds against it.  So stop sweating the small stuff… remember that you are a Black Swan.

 

BOOLE MICROCAP FUND

An equal weighted group of micro caps generally far outperforms an equal weighted (or cap-weighted) group of larger stocks over time.  See the historical chart here:  https://boolefund.com/best-performers-microcap-stocks/

This outperformance increases significantly by focusing on cheap micro caps.  Performance can be further boosted by isolating cheap microcap companies that show improving fundamentals.  We rank microcap stocks based on these and similar criteria.

There are roughly 10-20 positions in the portfolio.  The size of each position is determined by its rank.  Typically the largest position is 15-20% (at cost), while the average position is 8-10% (at cost).  Positions are held for 3 to 5 years unless a stock approaches intrinsic value sooner or an error has been discovered.

The mission of the Boole Fund is to outperform the S&P 500 Index by at least 5% per year (net of fees) over 5-year periods.  We also aim to outpace the Russell Microcap Index by at least 2% per year (net).  The Boole Fund has low fees.

 

If you are interested in finding out more, please e-mail me or leave a comment.

My e-mail: jb@boolefund.com

 

 

 

Disclosures: Past performance is not a guarantee or a reliable indicator of future results. All investments contain risk and may lose value. This material is distributed for informational purposes only. Forecasts, estimates, and certain information contained herein should not be considered as investment advice or a recommendation of any particular security, strategy or investment product. Information contained herein has been obtained from sources believed to be reliable, but not guaranteed. No part of this article may be reproduced in any form, or referred to in any other publication, without express written permission of Boole Capital, LLC.

The Man Who Solved the Market

September 18, 2022

Gregory Zuckerman’s new book, The Man Who Solved the Market: How Jim Simons Launched the Quant Revolution, is a terrific book.

When Zuckerman set out to write a book about Jim Simons’s Renaissance Technologies, Simons and others made it clear to Zuckerman not to expect any help from them.  Zuckerman wasn’t surprised.  He knew that Simons and team were among the most secretive traders in Wall Street history.  Zuckerman writes:

There were compelling reasons I was determined to tell Simons’s story.  A former math professor, Simons is arguably the most successful trader in the history of modern finance.  Since 1988, Renaissance’s flagship Medallion hedge fund has generated average annual returns of 66 percent… No one in the investment world comes close.  Warren Buffett, George Soros, Peter Lynch, Steve Cohen, and Ray Dalio all far short…

Zuckerman adds:

Simons’s pioneering methods have been embraced in almost every industry, and reach nearly every corner of everyday life.  He and his team were crunching statistics, turning tasks over to machines, and relying on algorithms more than three decades ago—long before these tactics were embraced in Silicon Valley, the halls of government, sports stadiums, doctors’ offices, military command centers, and pretty much everywhere else forecasting is required.

With persistence, Zuckerman ended up doing over four hundred interviews with more than thirty current and former Renaissance employees.  And he did interviews with a larger number of friends, family members, and others.  Zuckerman:

I owe deep gratitude to each individual who spent time sharing memories, observations, and insights.  Some accepted substantial personal risk to help me tell this story.  I hope I rewarded their faith.

(Jim Simons, by Gert-Martin Greuel, via Wikimedia Commons)

 

Part One: Money Isn’t Everything

CHAPTER ONE

In the winter of 1952, Jimmy Simons was fourteen years old.  He was trying to earn spending money at Breck’s garden supply near Newton, Massachusetts, where his home was.  Jimmy was absent-minded and misplaced almost everything.  So the owners asked him to sweep the floor.  A few weeks later, having finished the Christmas-time job, the owners asked Jimmy what he wanted to do.  He replied:

I want to study mathematics at MIT.

Breck’s owners burst out laughing.  How could someone so absent-minded study mathematics?  And at MIT?  Zuckerman writes:

The skepticism didn’t bother Jimmy, not even the giggles.  The teenager was filled with preternatural confidence and an unusual determination to accomplish something special, the result of supportive parents who had experienced both high hopes and deep regrets in their own lives.

Jimmy remained an only child of Marcia and Matthew Simons after Marcia endured a series of miscarriages.

A sharp intellect with an outgoing personality and subtle wit, Marcia volunteered in Jimmy’s school but never had the opportunity to work outside the home.  She funneled her dreams and passions into Jimmy, pushing him academically and assuring him that success was ahead.

Matty Simons was a sales manager for 20th Century Fox.  He loved the job.  But then his father-in-law, Peter Kantor, asked him to work at his shoe factory.  Peter promised Matty an ownership stake.  Matty felt obliged to join the family business.  Zuckerman:

Matty Simons spent years as the general manager of the shoe factory, but he never received the ownership share Peter had promised.  Later in life, Matty told his son he wished he hadn’t forgone a promising and exciting career to do what was expected of him.

Simons says the lesson he learned was to do what you like in life, not what you think you ‘should’ do.  Simons never forgot this lesson.

Even as a young kid, Simons loved to think, often about mathematics.  Zuckerman:

Unlike his parents, Jimmy was determined to focus on his own passions.  When he was eight, Dr. Kaplan, the Simons family doctor, suggested a career in medicine, saying it was the ideal profession “for a bright Jewish boy.”

Jimmy bristled.

“I want to be a mathematician or a scientist,” he replied.

The doctor tried to reason with the boy.  “Listen, you can’t make any money in mathematics.”

Jimmy said he wanted to try.

Zuckerman continues:

He loved books, frequently visiting a local library to take out four a week, many well above his grade level.  Mathematical concepts captivated him most, however.

After finishing high school in three years, the seventeen-year-old took a trip across the country with a friend.  One thing they encountered was severe poverty, which made them more sensitive to the predicaments of society’s disadvantaged.

Soon Simons enrolled at MIT.

(MIT logo, via Wikimedia Commmons)

He skipped the first year of mathematics because he had taken advanced-placement courses in high school.  Zuckerman:

Overconfident during the second semester of his freshman year, Simons registered for a graduate course in abstract algebra.  It was an outright disaster.  Simons was unable to keep up with the classmates and couldn’t understand the point of the assignments and course topics.

Simons bought a book on the subject and took it home for the summer, reading and thinking for hours at a time.  Finally, it clicked.  Simons aced subsequent algrebra classes.  Though he received a D in an upper-level calculus course in his sophomore year, the professor allowed him to enroll in the next level’s class, which discussed Stokes’ theorem, a generalization of Isaac Newton’s fundamental theorem of calculus that relates line integrals to surface integrals in three dimensions.  The young man was fascinated—a theorem involving calculus, algebra, and geometry seemed to produce simple, unexpected harmony.  Simons did so well in the class that students came to him seeking help.

“I just blossomed,” Simons says.  “It was a glorious feeling.”

Simons loved the beauty of mathematics.  Although Simons realized he wasn’t quite the best, he had an imaginative approach to problems and an instinct to focus on the kinds of problems that might lead to breakthroughs.

When Simons returned to MIT to begin his graduate studies, his advisor suggested he finish his PhD at the University of California, Berkeley, so he could work with a professor named Shiing-Shen Chern, a former math prodigy from China and a leading differential geometer and topologist.

Meanwhile, he had met an eighteen-year-old named Barbara Bluestein.  They talked a great deal and eventually decided to get engaged.  Over strong objections from her parents, Barbara decided to go with Simons to Berkeley.  The pair got married in Reno, Nevada.  Simons used the money they had left after the marriage to play poker.  He won enough to buy Barbara a black bathing suit.

Back at Berkeley:

…Simons made progress on a PhD dissertation focused on differential geometry—the study of curved, multidimensional spaces using methods from calculus, topology, and linear algebra.  Simons also spent time on a new passion: trading.  The couple had received $5,000 as a wedding gift, and Simons was eager to multiply the cash.

Simons bought a couple of stocks.  But they didn’t move and he asked a broker if they had anything “more exciting.”  The broker suggested soybeans.

Simons knew nothing about commodities or how to trade futures… but he became an eager student.  At the time, soybeans sold for $2.50 per bushel.  When the broker said Merrill Lynch’s analysts expected prices to go to three dollars or even higher, Simons’s eyes widened.  He bought two futures contracts, watched soybeans soar, and scored several thousand dollars of profits in a matter of days.

Simons was hooked.

“I was fascinated by the action and the possibility I could make money short-term,” he says.

After a respected journal published Simons’s dissertation, he won a prestigious, three-year teaching position at MIT.  However, Simons began to worry that his whole life would be research and teaching.

“Is this it?  Am I going to do this my whole life?” he asked Barbara one day at home.  “There has to be more.”

In 1963, Simons accepted a research position at Harvard.

 

CHAPTER TWO

Zuckerman writes:

In 1964, Simons quit Harvard University to join an intelligence group helping to fight the ongoing Cold War with the Soviet Union.  The group told Simons he could continue his mathematics research as he worked on government assignments.  Just as important, he doubled his previous salary and began paying off his debts.

Simons’s offer came from the Princeton, New Jersey, division of the Institute for Defense Analyses, an elite research organization that hired mathematicians from top universities to assist the National Security Agency—the United States’ largest and most secretive intelligence agency—in detecting and attacking Russian codes and ciphers.

…The IDA taught Simons how to develop mathematical models to discern and interpret patterns in seemingly meaningless data.  He began using statistical analysis and probability theory, mathematical tools that would influence his work.

Simons learned that he liked making algorithms and testing things out on a computer.  Simons became a sleuthing star.

Illustration by Stuart Miles

Also, Simons learned by seeing how the group recruited new researchers.  The recruits were identified by brainpower, creativity, and ambition, rather than for any particular expertise or background.  Simons met Lenny Baum, one of the most-accomplished code breakers.  Baum developed a saying that became the group’s credo:

“Bad ideas is good, good ideas is terrific, no ideas is terrible.”

Zuckerman notes:

The team routinely shared credit and met for champagne toasts after discovering solutions to particularly thorny problems.  Most days, researchers wandered into one another’s offices to offer assistance or lend an ear.  When staffers met each day for afternoon tea, they discussed the news, played chess, worked on puzzles, or competed at Go, the complicated Chinese board game.

Simons and his wife threw regular dinner parties at which IDA staffers became inebriated on Barbara’s rum-heavy Fish House Punch.  The group played high-stakes poker matches that lasted until the next morning, with Simons often walking away with fistfuls of his colleagues’ cash.

Photo by Gadosp

Meanwhile, Simons was making progress in his research on minimal varieties, a subfield of differential geometry in which he long had an interest.  Zuckerman:

He was hoping to discover and codify universal principles, rules, and truths, with the goal of furthering the understanding of these mathematical objects.  Albert Einstein argued that there is a natural order in the world; mathematicians like Simons can be seen as searching for evidence of that structure.  There is true beauty to their work, especially when it succeeds in revealing something about the universe’s natural order.  Often, such theories find practical applications, even many years later, while advancing our knowledge of the universe.

Eventually, a series of conversations with Frederick Almgren Jr., a professor at nearby Princeton University who had solved the problem in three dimensions, helped Simons achieve a breakthrough.  Simons created a partial differential equation of his own, which became known as the Simons equation, and used it to develop a uniform solution through six dimensions…

In 1968, Simons published “Minimal Varieties in Riemannian Manifolds,” which became a foundational paper for geometers, proved crucial in related fields, and continues to garner citations, underscoring its enduring significance.  These achievements helped establish Simons as one of the world’s preeminent geometers.

At the same time, Simons was studying the stock market in an effort to figure out how to make money.  Collaborating with Baum and two other colleagues, Simons developed a stock-trading system.  Simons and his colleagues ignored fundamental information such as earnings, dividends, and corporate news.  Instead, they searched for “macroscopic variables” that could predict the short-term behavior of the stock market.

Here’s what was really unique: [They] didn’t try to identify or predict these states using economic theory or other conventional methods, nor did the researchers seek to address why the market entered certain states.  Simons and his colleagues used mathematics to determine the set of states best fitting the observed pricing data; their model then made its bets accordingly.  The whys didn’t matter, Simons and his colleagues seemed to suggest, just the strategies to take advantage of the inferred states.

Simons and his colleagues used a mathematical tool called a hidden Markov model.

(Hidden Markov models, via Wikimedia Commons)

Also, they weren’t alone.  For instance, mathematician Ed Thorp developed an early form of computerized trading.

***

Simons was really good at identifying the most promising ideas of his colleagues.  Soon, he was in line to become deputy director of IDA.  Then came the Vietnam war.  Simons wrote a six-paragraph letter to The New York Times, arguing that there were better uses of the nation’s resources than the Vietnam war.  As a result, IDA fired Simons.

A friend asked Simons what his ideal job was.  Simons said he’d like to be the chair of a large math department, but he was too young and didn’t know the right people.  The friend told Simons that he had an idea.  Soon thereafter, Simons received a letter from John Toll, president of SUNY Stony Brook, a public university on Long Island.  The school had already spent five years looking for someone to chair its math department.

Toll was leading a $100 million, government-funded push to make SUNY Stony Brook the “Berkeley of the East.”

In 1968, Simons moved his family to Long Island and he began recruiting.  One person he targeted was a Cornell University mathematician named James Ax, who had won the prestigious Cole Prize in number theory.  Simons and Ax had been friendly as graduate students at Berkeley.  Simons charmed Ax into moving to Stony Brook.  Zuckerman:

Ax’s decision sent a message that Simons meant business.  As he raided other schools, Simons refined his pitch, focusing on what it might take to lure specific mathematicians.  Those who valued money got raises; those focused on personal research got lighter class loads, extra leave, generous research support, and help evading irritating administrative requirements.

Meanwhile, Simons’s marriage to Barbara was struggling.  They eventually divorced.  Barbara went on to earn a PhD in computer science at Berkeley in 1981.  Subsequently, she had a distinguished career.  Asked about her marriage to Simons, she reflected that they had gotten married when they were too young.

Now alone on Long Island, Simons sought a nanny to help when his three children were at his house.  That’s how he met Marilyn Hawrys, a twenty-two-year-old blond who later became a graduate student in economics at Stony Brook.  Simons asked her on a date, and they started seeing each other.

Around this time, Simons made a breakthrough with Shiing-Shen Chern.

On his own, Simons made a discovery related to quantifying shapes in curved, three-dimensional spaces.  He showed his work to Chern, who realized the insight could be extended to all dimensions.  In 1974, Chern and Simons published “Characteristic Forms and Geometric Invariants,” a paper that introduced Chern-Simons invariants… which proved useful in various aspects of mathematics.

In 1976, at the age of thirty-seven, Simons was awarded the American Mathematical Society’s Oswald Veblen Prize in Geometry, the highest honor in the field, for his work with Chern and his earlier research in minimal varieties.  A decade later, theoretical physicist Edward Witten and others would discover that Chern-Simons theory had applications to a range of areas in physics, including condensed matter, string theory, and supergravity.  It even became crucial to methods used by Microsoft and others in their attempts to develop quantum computers capable of solving problems vexing modern computers, such as drug development and artificial intelligence.  By 2019, tens of thousands of citations in academic papers—approximately three a day—referenced Chern-Simons theory, cementing Simons’s position in the upper echelon of mathematics and physics.

Simons was ready for a new challenge.  He had recently invested money with Charlie Freifeld, who had taken a course from Simons at Harvard.  Freifeld used econometric models to predict the prices of commodities.  Soon Simons’s investment with Freifeld had increased tenfold.  This got Simons’s excited again about the challenge of investing.

In 1978, Simons left academia to start an investment firm focused on currency trading.  (World currencies had recently been allowed to float.)  Some academics thought that Simons was squandering a rare talent.

 

CHAPTER THREE

Early summer of 1978, a few miles down the road from Stony Brook University:

Simons sat in a storefront office in the back of a dreary strip mall.  He was next to a woman’s clothing boutique, two doors down from a pizza joint, and across from the tiny, one-story Stony Brook train station.  His space, built for a retail establishment, had beige wallpaper, a single computer terminal, and spotty phone service.  From his window, Simons could barely see the aptly named Sheep Pasture Road, an indication of how quickly he had gone from broadly admired to entirely obscure.

The odds weren’t in favor of a forty-year-old mathematician embarking on his fourth career, hoping to revolutionize the centuries-old world of investing.

Simons hadn’t shown any real talent in investing.  He acknowledged that his investment with Freifeld had been “completely lucky.”

Photo by Ulrich Willmunder

They only held on to the profits because they had agreed to cash out if they ever made a large amount.  Weeks after they sold their position, sugar prices plummeted, which neither Freifeld nor Simons had predicted.  They had barely avoided disaster.  Zuckerman:

Somehow, Simons was bursting with self-confidence.  He had conquered mathematics, figured out code-breaking, and built a world-class university department.  Now he was sure he could master financial speculation, partly because he had developed a special insight into how financial markets operated.

[…]

It looks like there’s some structure here, Simons thought.

He just had to find it.

Simons decided to treat financial markets like any other chaotic system.  Just as physicists pore over vast quantities of data and build elegant models to identify laws in nature, Simons would built mathematical models to identify order in financial markets.  His approach bore similarities to the strategy he had developed years earlier at the Institute for Defense Analyses, when he and his colleagues wrote the research paper that determined that markets existed in various hidden states that could be identified with mathematical models.  Now Simons would test the approach in real life.

Simons named his company Monemetrics, combining “money” and “econometrics.”  He then began a process he knew well: hiring a team of big brains.  Zuckerman:

He did have an ideal partner in mind for his fledgling firm:  Leonard Baum, one of the coauthors of the IDA research paper and a mathematician who had spent time discerning hidden states and making short-term predictions in chaotic environments.  Simons just had to convince Baum to risk his career on Simons’s radical, unproven approach.

Baum’s parents had fled Russia for Brooklyn to escape poverty and anti-Semitism.  In high school, Baum was six feet tall and his school’s top sprinter.  He also played tennis.  Baum graduated Harvard University in 1953 and then earned a PhD in mathematics.  Zuckerman:

After joining the IDA in Princeton, Baum was even more successful breaking code than Simons, receiving credit for some of the unit’s most important, and still classified, achievements.

[…]

Balding and bearded, Baum pursued math research while juggling government assignments, just like Simons.  Over the course of several summers in the late 1960s, Baum and Lloyd Welch, an information theorist working down the hall, developed an algorithm to analyze Markov chains, which are sequences of events in which the probability of what happens next depends only on the current state, not past events.  In a Markov chain, it is impossible to predict future steps with certainty, yet one can observe the chain to make educated guesses about possible outcomes…

A hidden Markov process is one in which the chain of events is governed by unknown, underlying parameters or variables.  One sees the results of the chain but not the “states” that help explain the progression of the chain… Some investors liken financial markets, speech recognition patterns, and other complex chains of events to hidden Markov models.

The Baum-Welch algorithm provided a way to estimate probabilities and parameters within these complex sequences with little more information than the output of the processes…

Baum usually minimized the importance of his accomplishment.  Today, though, Baum’s algorithm, which allows a computer to teach itself states and probabilities, is seen as one of the twentieth century’s notable advances in machine learning, paving the way for breakthroughs affecting the lives of millions in fields from genomics to weather prediction.  Baum-Welch enabled the first effective speech recognition system and even Google’s search engine.

Zuckerman again:

Baum began working with Simons once a week.  By 1979, Baum, then forty-eight years old, was immersed in trading, just as Simons had hoped.  A top chess player in college, Baum felt he had discovered a new game to test his mental faculties.  He received a one-year leave of absence from the IDA and moved his family to Long Island and a rented, three-bedroom Victorian house lined with tall bookcases…

It didn’t take Baum long to develop an algorithm directing Monemetrics to buy currencies if they moved a certain level below their recent trend line and sell if they veered too far above it.  It was a simple piece of work, but Baum seemed on the right path, instilling confidence in Simons.

Zuckerman adds:

Baum became so certain their approach would work, and so hooked on investing, that he quit the IDA to work full-time with Simons.

Baum would receive 25 percent of the company’s profits.  Simons would test strategies in Monemetrics.  If they worked, he would implement them in a limited investment partnership he launched called Limroy, which included money from outside investors.  Simons had tried to raise $4 million and had gotten close enough that he felt ready to launch.  Zuckerman:

To make sure he and Baum were on the right track, Simons asked James Ax, his prized recruit at Stony Brook, to come by and check out their strategies.  Like Baum a year or so earlier, Ax knew little about investing and cared even less.  He immediately understood what his former colleagues were trying to accomplish, though, and became convinced they were onto something special.  Not only could Baum’s algorithm succeed in currencies, Ax argued, but similar predictive models could be developed to trade commodities, such as wheat, soybeans, and crude oil.  Hearing that, Simons persuaded Ax to leave academia, setting him up with his own trading account.  Now Simons was really excited.  He had two of the most acclaimed mathematicians working with him to unlock the secrets of the markets and enough cash to support their efforts.

One day Baum realized that Margaret Thatcher was keeping the British pound at an unsustainably low level.

Illustration by Tatiana Muslimova

Overcome with excitement, he rushed to the office to tell Simons.  They started buying as much as they could of the British pound, which shot up in value.  They then made accurate predictions for the Japanese Yen, West German deutsche mark, and Swiss franc.  The fund made tens of millions of dollars.  Zuckerman:

After racking up early currency winnings, Simons amended Limroy’s charter to allow it to trade US Treasury bond futures contracts as well as commodities.  He and Baum—who now had their own, separate investment accounts—assembled a small team to build sophisticated models that might identify profitable trades in currency, commodity, and bond markets.

Simons was having a great time, but the fun wouldn’t last.

***

Simons needed someone to program their computers.  He discovered Greg Hullender, a nineteen-year-old student at the California Institute of Technology.  Simons offered Hullender $9,000 a year plus a share of the firm’s profits.

Limroy proceeded to lose money for the next six months.  Simons was despondent.  At one point he told Hullender, “Sometimes I look at this and feel I’m just some guy who doesn’t really know what he’s doing.”  Zuckerman:

In the following days, Simons emerged from his funk, more determined than ever to build a high-tech trading system guided by algorithms, or step-by-step computer instructions rather than human judgment.

[…]

The technology for a fully automated system wasn’t there yet, Simons realized, but he wanted to try some more sophisticated methods.

Simons proceeded to launch a project of gathering as much past data as possible.  He got commodity, bond, and currency prices going back decades, even before World War II in some cases.  Zuckerman:

Eventually, the group developed a system that could dictate trades for various commodity, bond, and currency markets.

The system produced automated trade recommendations, a step short of automated trades, but the best they could do then.  But soon Simons and Baum lost confidence in their system.  They couldn’t understand why the system was making certain recommendations, and also they had another losing streak.

Simons and Baum drifted towards a more traditional investing approach.  They looked for undervalued investments and also tried to get the news faster than others in order to react to it before others.  They were investing about $30 million at that point.  Zuckerman writes:

Their traditional trading approach was going so well that, when the boutique next door closed, Simons rented the space and punched through the adjoining wall.  The new space was filled with offices for new hires, including an economist and others who provided expert intelligence and made their own trades, helping to boost returns.  At the same time, Simons was developing a new passion: backing promising technology companies, including an electronic dictionary company called Franklin Electronic Publishers, which developed the first hand-held computer.

In 1982, Simons changed Monemetrics’ name to Renaissance Technologies Corporation, reflecting his developing interest in these upstart companies.  Simons came to see himself as a venture capitalist as much as a trader.

Meanwhile, Baum excelled by relying on his own research and intuition:

He was making so much money trading various currencies using intuition and instinct that pursuing a systematic, “quantitative” style of trading seemed a waste of time.  Building formulas was difficult and time-consuming, and the gains figured to be steady but never spectacular.  By contrast, quickly digesting the office’s news ticker, studying newspaper articles, and analyzing geopolitical events seemed exciting and far more profitable.

Between July 1979 and March 1982, Baum made $43 million in profits, almost double his original stake from Simons.

Baum tended to hold on to his investments when he thought that a given trend would continue.  Eventually this caused a rift between Baum and Simons.  For example, in the fall of 1979, both Baum and Simons bought gold around $250 an ounce.

(Photo by Daniel Schreurs)

By January 1980, gold had soared past $700.  Simons sold his position, but Baum held on thinking the trend would continue.  Around this time, Simons learned that people were lining up to sell their physical gold—like jewelry—to take advantage of the high prices.  Simons grew concerned that the increase in supply could crush gold prices.

When he got back in the office, Simons ordered Baum to sell.  Baum refused.  Baum was sitting on more than $10 million in profits and gold had shot past $800 an ounce.  Baum was driving Simons crazy.  Finally Simons called the firm’s broker and put the phone to Baum’s ear.  Simons ordered Baum: “Tell him you’re selling.”  Baum finally capitulated.  Within months, gold shot past $865 an ounce and Baum was complaining that Simons had cost him serious money.  But just a few months later, gold was under $500.

In 1983, Federal Reserve Chair Paul Volcker predicted a decline in interest rates, and inflation appeared to be under control.  Baum purchased tens of millions of dollars of US bonds.  However, panic selling overcame the bond market in the late spring of 1984 amid a large increase in bond issuance by the Reagan administration.

Once again, Simons told Baum to lighten up, but Baum refused.  Baum also had a huge bet that the Japanese Yen would continue to appreciate.  That bet was also backfiring.  Zuckerman:

When the value of Baum’s investment positions had plummeted 40 percent, it triggered an automatic clause in his agreement with Simons, forcing Simons to sell all of Baum’s holdings and unwind their trading affiliation, a sad denouement to a decades-long relationship between the esteemed mathematicians.

Ultimately, Baum was very right about US bonds.  By then, Baum was only trading for himself.  Baum also returned to Princeton.  He was now sleeping better and he had time for mathematics.  Baum focused on prime numbers and the Riemann hypothesis.  Also, for fun, he traveled the United States and competed in Go tournaments.

Meanwhile, Simons was upset about the losses.  He considered just focusing on technology investing.  He gave clients an opportunity to withdraw, but most kept faith that Simons would figure out a way to improve results.

 

CHAPTER FOUR

Simons came to the conclusion that Baum’s approach, using intellect and instinct, was not a reliable way to make money.  Simons commented: “If you make money, you feel like a genius.  If you lose, you’re a dope.”  Zuckerman:

Simons wondered if the technology was yet available to trade using mathematical models and preset algorithms, to avoid the emotional ups and downs that come with betting on markets with only intelligence and intuition.  Simons still had James Ax working for him, a mathematician who seemed perfectly suited to build a pioneering computer trading system.  Simons resolved to back Ax with ample support and resources, hoping something special would emerge.

Zuckerman continues:

In 1961, Ax earned a PhD in mathematics from the University of California, Berkeley, where he became friends with Simons, a fellow graduate student.  Ax was the first to greet Simons and his wife in the hospital after Barbara gave birth to their first child.  As a mathematics professor at Cornell University, Ax helped develop a branch of pure mathematics called number theory.  In the process, he forged a close bond with a senior, tenured academic named Simon Kochen, a mathematical logician.  Together, the professors tried to prove a famous fifty-year-old conjecture made by the famed Austrian mathematician Emil Artin, meeting immediate and enduring frustration.  To blow off steam, Ax and Kochen initiated a weekly poker game with colleagues and others in the Ithaca, New York, area.  What started as friendly get-togethers, with winning pots that rarely topped fifteen dollars, grew in intensity until the men fought over stakes reaching hundreds of dollars.

[…]

Ax spent the 1970s searching for new rivals and ways to best them.  In addition to poker, he took up golf and bowling, while emerging as one of the nation’s top backgammon players.

In 1979, Ax joined Simons.  First Ax looked at fundamentals.  Zuckerman:

Ax’s returns weren’t remarkable, so he began developing a trading system to take advantage of his math background.  Ax mined the assorted data Simons and his team had collected, crafting algorithms to predict where various currencies and commodities were headed.

[…]

Ax’s predictive models had potential, but they were quite crude.  The trove of data Simons and others had collected proved of little use, mostly because it was riddled with errors and faulty prices.  Also, Ax’s trading system wasn’t in any way automated—his trades were made by phone, twice a day, in the morning and at the end of the trading day.

Ax soon began to rely on a former professor:  Sandor Straus earned a PhD in mathematics from Berkeley in 1972 and went to teach at Stony Brook.  Straus thrived.  And he wore a long ponytail with John Lennon-style glasses.  In 1976, Straus joined Stony Brook’s computer center.  He helped Ax and others develop computer simulations.  But Straus wasn’t happy and was worried about money.  Simons offered to double Straus’s salary if he joined Monemetrics as a computer specialist.

Against the consistent advice of his father and friends, Straus eventually decided to join Monemetrics.  Straus began collecting a wide range of data.  Zuckerman:

No one had asked Straus to track down so much information.  Opening and closing prices seemed sufficient to Simons and Ax.  They didn’t even have a way to use all the data Straus was gathering, and with computer-processing power still limited, that didn’t seem likely to change.  But Straus figured he’d continue collecting the information in case it came in handy down the road.

Straus became somewhat obsessive in his quest to locate pricing data before others realized its potential value.  Straus even collected information on stock trades, just in case Simons’s team wanted it at some point in the future.  For Straus, gathering data became a matter of personal pride.

Zuckerman adds:

…No one had told Straus to worry so much about the prices, but he had transformed into a data purist, foraging and cleaning data the rest of the world cared little about.

…Some other traders were gathering and cleaning data, but no one collected as much as Strauss, who was becoming something of a data guru.

Straus’s data helped Ax improve his trading results…

Simons asked Henry Laufer, another Stony Brook mathematician, to join Monemetrics.  Laufer agreed.  Zuckerman:

Laufer created computer simulations to test whether certain strategies should be added to their trading model.  The strategies were often based on the idea that prices tend to revert after an initial move higher or lower.  Laufer would buy futures contracts if they opened at unusually low prices compared with their previous closing price, and sell if prices began the day much higher than their previous close.  Simons made his own improvements to the evolving system, while insisting that the team work together and share credit.

In 1985, Ax moved to Huntington Beach, California.  Ax and Straus established a new company: Axcom Limited.  Simons would get 25 percent of the profits, while Ax and Straus split the remaining 75 percent.  Laufer didn’t want to move west, so he returned to Stony Brook.  Zuckerman writes:

By 1986, Axcom was trading twenty-one different futures contracts, including the British pound, Swiss franc, deutsche mark, Eurodollars, and commodities including wheat, corn, and sugar.  Mathematical formulas developed by Ax and Straus generated most of the firm’s moves, though a few decisions were based on Ax’s judgment calls.  Before the beginning of trading each day, and just before the end of trading in the late afternoon, a computer program would send an electronic message to Greg Olsen, their broker at an outside firm, with an order and some simple conditions.  One example: “If wheat opens above $4.25, sell 36 contracts.”

However, Simons and the team were not finding new ways to make money, nor were they improving on their existing methods, which allowed rivals to catch up.  Zuckerman:

Eventually, Ax decided they needed to trade in a more sophisticated way.  They hadn’t tried using more-complex math to build trading formulas, partly because the computing power didn’t seem sufficient.  Now Ax thought it might be time to give it a shot.

Ax had long believed financial markets shared characteristics with Markov chains, those sequences of events in which the next event is only dependent on the current state.  In a Markov chain, each step along the way is impossible to predict with certainty, but future steps can be predicted with some degree of accuracy if one relies on a capable model…

To improve their predictive models, Ax concluded it was time to bring in someone with experience developing stochastic equations, the broader family of equations to which Markov chains belong.  Stochastic equations model dynamic processes that evolve over time and can involve a high level of uncertainty.

Soon Rene Carmona, a professor at University of California, Irvine, got a call from a friend who told him a group of mathematicians was looking for someone with his specialty—stochastic differential equations.  Simons, Ax, and Straus were interested.  Zuckerman:

Perhaps by hiring Carmona, they could develop a model that would produce a range of likely outcomes for their investments, helping to improve their performance.

Carmona was eager to lend a hand—he was consulting for a local aerospace company at the time and liked the idea of picking up extra cash working for Axcom a few days a week.  The challenge of improving the firm’s trading results also intrigued him.

“The goal was to invent a mathematical model and use it as a framework to infer some consequences and conclusions,” Carmona says.  “The name of the game is not to always be right, but to be right often enough.”

In 1987, after having made little progress, Carmona decided to spend the summer working full-time for Axcom.  Yet Carmona still couldn’t generate useful results.  Carmona soon realized that they needed regressions that could capture nonlinear relationships in market data.  Zuckerman explains:

He suggested a different approach.  Carmona’s idea was to have computers search for relationships in the data Strauss had amassed.  Perhaps they could find instances in the remote past of similar trading environments, then they could examine how prices reacted.  By identifying comparable trading situations and tracking what subsequently happened to prices, they could develop a sophisticated and accurate forecasting model capable of detecting hidden patterns.

For this approach to work, Axcom needed a lot of data, even more than Strauss and the others had collected.  To solve the problem, Strauss began to model data rather than just collect it.  In other words, to deal with gaps in historical data, he used computer models to make educated guesses as to what was missing.  They didn’t have extensive cotton pricing data from the 1940s, for example, but maybe creating the data would suffice…

Carmona suggested letting the model run the show by digesting all the various pieces of data and spitting out buy-and-sell decisions.  In a sense, he was proposing an early machine-learning system.  The model would generate predictions for various commodity prices based on complex patterns, clusters, and correlations that Carmona and the others didn’t understand themselves and couldn’t detect with the naked eye.

Elsewhere, statisticians were using similar approaches—called kernel methods—to analyze patterns in data sets.

At first, Simons couldn’t get comfortable because he couldn’t understand why the model was reaching certain conclusions.  Carmona told Simons to follow the data.  Ax urged Simons to let the computers do it.

(Illustration by Dmitry Gorelkin)

Zuckerman:

When the Axcom team started testing the approach, they quickly began to see improved results.  The firm began incorporating higher dimensional kernel regression approaches, which seemed to work best for trending models…

Simons was convinced they could do even better.  Carmona’s ideas helped, but they weren’t enough.  Simons called and visited, hoping to improve Axcom’s performance, but he mostly served as the pool operator, finding wealthy investors for the fund and keeping them happy, while attending to the various technology investments that made up about half of the $100 million assets now held by the firm.

Seeking even more mathematical firepower, Simons arranged for a well-respected academic to consult with the firm.  That move would lay the groundwork for a historic breakthrough.

 

CHAPTER FIVE

Elwyn Berlekamp attended MIT.  In his senior year, Berlekamp won a prestigious math competition to become a Putnam Fellow.  While pursuing a PhD at MIT, he focused on electrical engineering, studying with Peter Elias and Claude Shannon.  One day Shannon pulled Berlekamp aside and told him it wasn’t a good time to invest in the stock market.  Berlekamp had no money, so he laughed.  Also, he thought investing was a game where rich people play and that it doesn’t do much to improve the world.

During the summers of 1960 and 1962, Berlekamp worked as a research assistant at Bell Laboratories research center in Murray Hill, New Jersey.  While there, he worked for John Larry Kelly, Jr.  Kelly was a brilliant physicist who had been a pilot in the US Navy in World War II.  Kelly smoked six packs of cigarettes a day and invented a betting system to bet on college and professional football.  Kelly invented the Kelly criterion.

The Kelly criterion can be written as follows (Zuckerman doesn’t mention this, but it’s worth noting):

    • F = p – [q/o]

where

    • F = Kelly criterion fraction of current capital to bet
    • o = Net odds, or dollars won per $1 bet if the bet wins (e.g., the bet may pay 5 to 1, meaning you win $5 per each $1 bet if the bet wins)
    • p = probability of winning
    • q = probability of losing = 1 – p

The Kelly criterion has a unique mathematical property: if you know the probability of winning and the net odds (payoff), then betting exactly the percentage determined by the Kelly criterion leads to the maximum long-term compounding of capital, assuming that you’re going to make a long series of bets.  Betting any percentage that is not equal to that given by the Kelly criterion will inevitably lead to lower compound growth over a long period of time.

Berlekamp finished his PhD at the University of California, Berkeley.  He became an assistant professor of electrical engineering.  Zuckerman:

Berlekamp became an expert in decoding digital information, helping NASA decipher images coming back from satellites exploring Mars, Venus, and other parts of the solar system.  Employing principles he had developed studying puzzles and games… Berlekamp cofounded a branch of mathematics called combinatorial game theory and wrote a book called Algebraic Coding Theory, a classic in the field.

By the late 1960s, the Institute for Defense Analyses (IDA) hired Berlekamp.  He met Simons, but the two didn’t hit it off, despite having both spent time at MIT, Berkeley, and IDA.

“His mathematics were different from mine,” Berlekamp says.  “And Jim had an insatiable urge to do finance and make money.  He likes action… He was always playing poker and fussing around with the markets.”

In 1973, Berlekamp became the part owner of a cryptography company.  In 1985, Eastman Kodak acquired the company, giving Berlekamp an unexpected windfall.  Berlekamp complained that the money caused challenges in his marriage.  His wife wanted a bigger house, while he wanted to travel.

While trying to figure out how to invest the money, a friend told him to look at commodities.  This led Berlekamp to contact Simons, who told him, “I have just the opportunity for you.”  Berlekamp started flying to Huntington Beach a couple of times a month to learn to trade and to see if his expertise in statistical information theory might be useful.  Zuckerman says:

For all the brainpower the team was employing, and the help they were receiving from Carmona and others, Axcom’s model usually focused on two simple and commonplace trading strategies.  Sometimes, it chased prices, or bought various commodities that were moving higher or lower on the assumption that the trend would continue.

Photo by Sergio Delle Vedove

Other times, the model wagered that a price move was petering out and would reverse: a reversion strategy.

Ax had access to more extensive pricing information than his rivals, thanks to Straus’s growing collection of clean, historic data.  Since price movements often resembled those of the past, that data enabled the firm to more accurately determine when trends were likely to continue and when they were ebbing.  Computing power had improved and become cheaper, allowing the team to produce more sophisticated trading models, including Carmona’s kernel methods—the early, machine-learning strategy that had made Simons so uncomfortable.  With those advantages, Axcom averaged annual gains of about 20 percent, topping most rivals.

Yet Simons kept asking why returns weren’t better.  Adding to the tension, their rivals were multiplying.

With the Kelly criterion in mind, Berlekamp told Ax that when the odds were higher, they should make bigger bets.  Ax said they would, but seemed noncommittal.  Zuckerman:

Berlekamp discovered other problems with Axcom’s operations.  The firm traded gold, silver, copper, and other metals, as well as hogs and other meats, and grains and other commodities.  But their buy-and-sell orders were still placed through emailed instructions to their broker, Greg Olsen, at the open and close of each trading day, and Axcom often held on to investments for weeks or even months at a time.

That’s a dangerous approach, Berlekamp argued, because markets can be volatile.  Infrequent trading precluded the firm from jumping on new opportunities as they arose and led to losses during extended downturns.  Berlekamp urged Ax to look for smaller, short-term opportunities—get in and get out.  Ax brushed him off again, this time citing the cost of doing rapid trading.  Besides, Straus’s intraday price data was riddled with inaccuracies—he hadn’t fully “cleaned” it yet—so they couldn’t create a reliable model for short-term trades.

Meanwhile, some investors didn’t have faith in Simons’s venture-capital investments.  So Simons closed it down in March 1988 and opened, with Ax, an offshore hedge fund focused solely on trading.  They named the hedge fund Medallion because each had gotten a prestigious math award.

Within six months, Medallion was struggling.  Part of it seemed to be due to the fact that Ax had gotten less focused.  He moved to an isolated spot near Malibu and he rarely came in to the office even though he was managing nearly a dozen employees.  Soon Ax purchased a spectacular home on a cliff in Pacific Palisades.

At one point in 1989, Medallion was down nearly 30 percent from the middle of the previous year.  Simons ordered Axcom to halt all trading based on the firm’s struggling, longer-term predictive signals until Ax and his team developed a plan to improve results.  In the meantime, Ax was only allowed to do short-term trading, which was just 10 percent of the fund’s activity.

Ax thought Simons’s order violated their partnership agreement.  Ax was going to sue Simons.  Technically, however, Axcom was trading for a general partnership controlled by Simons.

Then Berlekamp offered to buy Ax’s stake.  Ax agreed.  The deal left Berlekamp with a 40 percent stake in the firm, while Straus and Simons had 25 percent each.  Ax still had 10 percent.  But he effectively retired from trading.  Ax moved to San Diego.  He wrote poetry, took screenwriting classes, and wrote a science-fiction thriller.

 

CHAPTER SIX

Berlekamp moved the firm close to his home in Berkeley.

The team forged ahead, with Berlekamp focused on implementing some of the most promising recommendations Ax had ignored.  Simons, exhausted from months of bickering with Ax, supported the idea.

“Let’s bank some sure things,” Berlekamp told Simons.

In addition to the issue of cost, short-term trading interested few investors because the gains were tiny.  Zuckerman:

Berlekamp hadn’t worked on Wall Street and was inherently skeptical of long-held dogmas developed by those he suspected weren’t especially sophisticated in their analysis.  He advocated for more short-term trades.  Too many of the firm’s long-term moves had been duds, while Medallion’s short-term trades had proved its biggest winners, thanks to the work of Ax, Carmona, and others.  It made sense to try to build on that success.  Berlekamp also enjoyed some good timing—by then, most of Straus’s intraday data had been cleaned up, making it easier to develop fresh ideas for shorter-term trades.

Their goal remained the same: scrutinize historic price information to discover sequences that might repeat, under the assumption that investors will exhibit similar behavior in the future.

Berlekamp also observed that doing a higher number of shorter-term trades meant that no individual trade could hurt results.  This reduces the portfolio’s risk.  Berlekamp and his colleagues view Medallion as being like a casino.

“If you trade a lot, you only need to be right 51 percent of the time,” Berlekamp argued to a colleague.  “We need a smaller edge on each trade.”

Zuckerman writes:

Simons and his researchers didn’t believe in spending much time proposing and testing their own intuitive trade ideas.  They let the data point them to the anomalies signaling opportunity.  They also didn’t think it made sense to worry about why these phenomena existed.  All that mattered was that they happened frequently enough to include in their updated trading system, and that they could be tested to ensure they weren’t statistical flukes.

Zuckerman again:

Beyond the repeating sequences that seemed to make sense, the system Berlekamp, Laufer, and Straus developed spotted barely perceptible patterns in various markets that had no apparent explanation.  These trends and oddities sometimes happened so quickly that they were unnoticeable to most investors.  They were so faint, the team took to calling them ghosts, yet they kept reappearing with enough frequency to be worth additions to their mix of trade ideas.  Simons had come around to the view that the whys didn’t matter, just that the trades worked.

[…]

By late 1989, after about six months of work, Berlekamp and his colleagues were reasonably sure their rebuilt trading system—focused on commodity, currency, and bond markets—could prosper.

[…]

The firm implemented its new approach in late 1989 with the $27 million Simons still managed.  The results were almost immediate, startling nearly everyone in the office.  They did more trading than ever, cutting Medallion’s average holding time to just a day and a half from a week and a half, scoring profits almost every day.

At one point, Medallion had every one of its positions with the Stotler Group, a commodity-trading firm run by Karsten Mahlmann.  There were rumors Stotler was in trouble.  Berlekamp wasn’t sure what to do.  But Simons was.  He ordered Berlekamp to move all their positions to a different broker.  So he did.  Soon Stotler filed for bankruptcy.  Zuckerman:

Simons and his firm had narrowly escaped a likely death blow.

***

Zuckerman writes:

For much of 1990, Simons’s team could do little wrong, as if they had discovered a magic formula after a decade of fumbling around in the lab.  Rather than transact only at the open and close of trading each day, Berlekamp, Laufer, and Strauss traded at noon, as well.  Their system became mostly short-term moves, with long-term trades representing about 10 percent of activity.

One day, Axcom made more than $1 million, a first for the firm.  Simons rewarded the team with champagne, much as the IDA’s staff had passed around flutes of bubbly after discovering solutions to thorny problems.

(Photo by William MacGregor)

Zuckerman:

Medallion scored a gain of 55.9 percent in 1990, a dramatic improvement on its 4 percent loss the previous year.  The profits were especially impressive because they were over and above the hefty fees charged by the fund, which amounted to 5 percent of all assets managed and 20 percent of all gains generated by the fund.

Simons was convinced that the team had discovered a highly profitable strategy.  Nonetheless, he kept messing around with trade ideas like gold.  Zuckerman:

Berlekamp was baffled.  It was Simons who had pushed to develop a computerized trading system free of human involvement, and it was Simons who wanted to rely on the scientific method, testing overlooked anomalies rather than using crude charts or gut instinct.  Berlekamp, Laufer, and the rest of the team had worked diligently to remove humans from the trading loop as much as possible.  Now Simons was saying he had a good feeling about gold prices and wanted to tweak the system?

[…]

“We must still rely on human judgment and manual intervention to cope with a drastic, sudden change,” [Simons] explained in a letter [to clients] that month.

Simons told Berlekamp how much better the fund should be doing.  Simons was convinced the $40 million fund could achieve remarkable success.  Berlekamp didn’t think the fund could do much better than it already had.  Eventually Berlekamp—who was enjoying teaching at Berkeley more than ever—suggested to Simons that Simons buy him out.  Zuckerman:

Which is exactly what Simons did.  In December 1990, Axcom was disbanded; Simons purchased Berlekamp’s ownership interest for cash, while Strauss and Ax traded their Axcom stakes for shares in Renaissance, which began to manage the Medallion fund.  Berlekamp returned to Berkeley to teach and do full-time math research, selling his Axcom shares at a price that amounted to six times what he had paid just sixteen months earlier…

“It never occurred to me that we’d go through the roof,” Berlekamp says.

 

CHAPTER SEVEN

Zuckerman writes:

Like the technical traders before him, Simons practiced a form of pattern analysis and searched for telltale sequences and correlations in market data.  He hoped to have a bit more luck than investors before him by doing his trading in a more scientific manner, however.  Simons agreed with Berlekamp that technical indicators were better at guiding short-term trades than long-term investments.  But Simons hoped rigorous testing and sophisticated predictive models, based on statistical analysis rather than eyeballing price charts, might help him escape the fate of the chart adherents who had crashed and burned.

Photo by Maxkabakov

But Simons didn’t realize that others were busy crafting similar strategies, some using their own high-powered computers and mathematical algorithms.  Several of these traders already had made enormous progress, suggesting that Simons was playing catch-up.

Zuckerman:

Edward Thorp became the first modern mathematician to use quantitative strategies to invest sizable sums of money.  Thorp was an academic who had worked with Claude Shannon, the father of information theory, and embraced the proportional betting system of John Kelly, the Texas scientist who had influenced Elwyn Berlekamp.

By the late 1980s, Thorp’s fund had $300 million under management, while Simons’s Medallion fund had only $25 million.  Thorp’s fund traded warrants, options, convertible bonds, and other derivative securities.

I wrote about Thorp’s autobiography, A Man for All Markets, here: https://boolefund.com/man-for-all-markets/

***

Gerry Bamburger, a computer-science graduate of Columbia University, gave technical support to Morgan Stanley’s stock traders.  Zuckerman:

Bamburger sensed opportunity.  If the bank created a database tracking the historic prices of various paired stocks, it could profit simply by betting on the return of these price-spreads to their historic levels after block trades or other unusual activity.  Bamburger’s bosses were swayed, setting him up with half a million dollars and a small staff.  Bamburger began developing computer programs to take advantage of “temporary blips” of paired shares… By 1985, he was implementing his strategy with six or seven stocks at a time, while managing $30 million, scoring profits for Morgan Stanley.

Morgan Stanley gave Bamburger a new boss: Nunzio Tartaglia, an astrophysicist.  This prompted Bamburger to quit Morgan Stanley and join Ed Thorp’s hedge fund.

Meanwhile, Tartaglia renamed his group Automated Proprietary Trading (APT).  Zuckerman:

New hires, including a former Columbia University computer-science professor named David Shaw and mathematician Robert Frey, improved profits.  The Morgan Stanley traders became some of the first to embrace the strategy of statistical arbitrage, or stat arb…. The team’s software ranked stocks by their gains or losses over the previous weeks, for example.  APT would then sell short, or bet against, the top 10 percent of the winners within an industry while buying the bottom 10 percent of the losers on the expectation that these trading patterns would revert.

Eventually, APT suffered losses.  Morgan Stanley ended up shutting the group down.  Zuckerman comments:

It wouldn’t be clear for many years, but Morgan Stanley had squandered some of the most lucrative trading strategies in the history of finance.

***

One of the Medallion fund’s chief competitors was a fund run by David Shaw, who had been a part of Morgan Stanley’s APT group.  Zuckerman:

Shaw, a supercomputing expert, hired math and science PhDs who embraced his scientific approach to trading.  He also brought on whip-smart employees from different backgrounds.  English and philosophy majors were among Shaw’s favorite hires, but he also hired a chess master, stand-up comedians, published writers, an Olympic-level fencer, a trombone player, and a demolitions specialist.

Soon Shaw’s fund became successful and was managing several hundred million dollars.  Jim Simons wasn’t sure what methods Shaw was using, but he did realize that he needed to hire more help in order to catch up.  Simons contacted Donald Sussman, a hedge-fund manager who had helped Shaw launch his fund.  Perhaps Sussman would give Simons a similar boost.

 

CHAPTER EIGHT

Zuckerman writes about Simons meeting with Sussman:

Simons had discarded a thriving academic career to do something special in the investing world.  But, after a full decade in the business, he was managing barely more than $45 million, a mere quarter the assets of Shaw’s firm.  The meeting had import—backing from Sussman could help Renaissance hire employees, upgrade technology, and become a force on Wall Street.

Simons’s presentation to Sussman went well.  But in the end, because Sussman was the sole source of capital for D. E. Shaw—which was generating 40 percent returns per year—Sussman decided not to invest in Renaissance.  Simons approached other potential backers, but no one would invest.  Most thought it was crazy to rely on trading models generated by computers.  Many also thought Simons’s fees were too high.  Simons charged 5 percent of assets plus 20 percent of profits, whereas most hedge funds at the time charged 2 percent of assets plus 20 percent of profits.  Perhaps most importantly, Renaissance had fewer than two years of impressive performance.

Simons knew he needed to hire more help.  He turned to the mathematician Henry Laufer.  Zuckerman:

Joining Stony Brook’s math department in 1971, Laufer focused on complex variables and algebraic geometry, veering away from classical areas of complex analysis to develop insights into more contemporary problems.

In 1992, Laufer joined Simons as a full-time employee.  Zuckerman:

Laufer made an early decision that would prove extraordinarily valuable: Medallion would employ a single trading model rather than maintain various models for different investments and market conditions, a style most quantitative firms would embrace.  A collection of trading models was simpler and easier to pull off, Laufer acknowledged.  But, he argued, a single model could draw on Straus’s vast trove of pricing data, detecting correlations, opportunities, and other signals across various asset classes.  Narrow, individual models, by contrast, can suffer from too little data.

Just as important, Laufer understood that a single, stable model based on some core assumptions about how prices and markets behave would make it easier to add new investments later on.

Zuckerman continues:

Simons wondered if there might be a better way to parse their data trove.  Perhaps breaking the day up into finer segments might enable the team to dissect intraday pricing information and unearth new, undetected patterns.  Laufer… eventually decided five-minute bars were the ideal way to carve things up.  Crucially, Straus now had access to improved computer-processing power, making it easier for Laufer to compare small slices of historic data…

Laufer’s five-minute bars gave the team the ability to identify new trends, oddities, and other phenomena, or, in their parlance, nonrandom trading effects.  Straus and others conducted tests to ensure they hadn’t mined so deeply into their data that they had arrived at bogus trading strategies, but many of the new signals seemed to hold up.

Zuckerman adds:

Simons was challenging them to solve yet another vexing problem: Given  the range of possible trades they had developed and the limited amount of money that Medallion managed, how much should the bet on each trade?  And which moves should they pursue and prioritize?  Laufer began developing a computer program to identify optimal trades throughout the day, something Simons began calling his betting algorithm.  Laufer decided it would be “dynamic,” adapting on its own along the way and relying on real-time analysis to adjust the fund’s mix of holdings given the probabilities of future market moves—an early form of machine learning.

Simons, with about a dozen employees, realized that he still needed to hire more people in order to take on D. E. Shaw and other top funds.  One person Simons hired was mathematician and programmer Nick Patterson.  His coding ability gave him an advantage over other mathematicians.  He graduated from the University of Cambridge.

(Photo by Cristian Saulean)

Patterson was a strong chess player and was also a stud at poker.

(Photo by Krastiu Vasilev)

After completing graduate school, Patterson worked as a cryptologist for the British government, where he made use of Bayes’s theorem of probability.

Because computing power was expanding exponentially, Patterson thought that Simons had a chance to revolutionize investing by applying high-level math and statistics.

Patterson began working on how to reduce trading costs:

…Laufer and Patterson began developing sophisticated approaches to direct trades to various futures exchanges to reduce the market impact of each trade.  Now Medallion could better determine which investments to pursue, a huge advantage as it began trading new markets and investments.  They added German, British, and Italian bonds, then interest-rate contracts in London, and, later, futures on Nikkei Stock Average, Japanese government bonds, and more.

The fund began trading more frequently.  Having first sent orders to a team of traders five times a day, it eventually increased to sixteen times a day, reducing the impact on prices by focusing on the periods when there was the most volume.

Medallion increased 71 percent in 1994.  This was especially impressive because the Federal Reserve had hiked interest rates several times, which led to losses for many investors.  Simons still didn’t know why their models were working.  He told a colleague: “I don’t know why planets orbit the sun.  That doesn’t mean I can’t predict them.”  Zuckerman:

At the time, most academics were convinced markets were inherently efficient, suggesting that there were no predictable ways to beat the market’s return, and that the financial decision-making of individuals was largely rational.  Simons and his colleagues sensed the professors were wrong.  They believed investors are prone to cognitive biases, the kinds that lead to panics, bubbles, boom, and busts.

Simons didn’t realize it, but a new strain of economics was emerging that would validate his instincts.  In the 1970s, Israeli psychologists Amos Tversky and Daniel Kahneman had explored how individuals make decisions, demonstrating how prone most are to act irrationally.  Later, economist Richard Thaler used psychological insights to explain anomalies in investor behavior, spurring the growth of the field of behavioral economics, which explored the cognitive biases of individuals and investors.

Link to my blog post on cognitive biases: https://boolefund.com/cognitive-biases/

(Illustration by Alain Lacroix)

Link to my blog post on the psychology of misjudgment: https://boolefund.com/the-psychology-of-misjudgment/

Simons saw how successful Medallion’s updated approach was, and he remembered how difficult it was for Baum, Ax, and himself to profit from instincts.  As a result, Simons committed to not overriding the model.  Medallion’s model profited from the errors and overreactions of other investors.  Medallion’s core assumption was that investors will behave in the future like they did in the past.

Other investors at long last began noticing Medallion’s stellar results.  Some of these investors made an investment in Medallion.  By that point, Medallion was sharing its track record, but it wasn’t sharing much about how their trading system worked because it didn’t want rivals to catch on.

By the end of 1993, Medallion had $280 million under management.  Simons grew concerned the they wouldn’t be as profitable if they got too big.  So he decided, for the time being, not to let new investors into the fund.  Furthermore, Simons got even more secretive about Medallion’s trading model.  Simons even pressured his investors not to divulge any information about the fund’s operations.

Zuckerman writes:

Medallion was still on a winning streak.  It was scoring big profits trading futures contracts and managed $600 million, but Simons was convinced the hedge fund was in a serious bind.  Laufer’s models, which measured the fund’s impact on the market with surprising precision, concluded that Medallion’s returns would wane if it managed much more money.  Some commodity markets, such as grains, were just too small to handle additional buying and selling by the fund without pushing prices around.  There were also limitations to how much more Medallion could do in bigger bond and currency markets.

Moreover:

Simons worried his signals were getting weaker as rivals adopted similar strategies.

Yet Simons continued to search for ways to grow the fund.  There was only one way to expand:  start investing in stocks.  The trouble was that Medallion had never developed any model to invest in stocks profitably.

Simons’s son Paul battled a birth disorder—ectodermal dysplasia.  Paul worked almost every day to strengthen his body, constantly doing push-ups and pull-ups.  Paul was also an accomplished skier and endurance bicycle rider.  One day Paul was taking a fast ride through Old Field Road in Setauket, near the home where he grew up.  An elderly woman backed out of her driveway, out of nowhere, and crushed Paul, killing him instantly.  Jim Simons and Barbara were completely devastated.

 

CHAPTER NINE

Zuckerman comments:

When it came to stocks, Simons seemed well out of his depth.

Most successful stock investors at the time focused on fundamentals, poring over financial statements in order to understand things such as assets, liabilities, sales, earnings, and cash flows.

Nick Patterson began a side job he liked: recruiting talent for Renaissance.  Zuckerman:

One day, after reading in the morning paper that IBM was slashing costs, Patterson became intrigued.  He was aware of the accomplishments of the computer giant’s speech-recognition group and thought their work bore similarity to what Renaissance was doing.  In early 1993, Patterson sent separate letters to Peter Brown and Robert Mercer, deputies of the group, inviting them to visit Renaissance’s offices to discuss potential positions.

Zuckerman explains that Robert Mercer’s passion for computers had been sparked by his father Thomas:

It began the very moment Thomas showed Robert the magnetic drum and punch cards of an IBM 650, one of the earliest mass-produced computers.  After Thomas explained the computer’s inner workings to his son, the ten-year-old began creating his own programs, filling up an oversize notebook.  Bob carried that notebook around for years before he ever had access to an actual computer.

At Sandia High School and the University of New Mexico, Mercer was a member of the chess, auto, and Russian clubs.  He was low-key, but he came alive for mathematics.  In 1964, he and two classmates won top honors in a national mathematics contest.  Zuckerman:

While studying physics, chemistry, and mathematics at the University of New Mexico, Mercer got a job at a weapons laboratory at the Kirkland Air Force Base eight miles away, just so he could help program the base’s supercomputer.

[…]

…Mercer spent the summer on the lab’s mainframe computer rewriting a program that calculated electromagnetic fields generated by nuclear fusion bombs.  In time, Mercer found ways to make the program one hundred times faster, a real coup.  Mercer was energized and enthused, but his bosses didn’t seem to care about his accomplishment.  Instead of running the old computations at the new, faster speed, they instructed Mercer to run computations that were one hundred times the size.  It seemed Mercer’s revved-up speed made little difference to them, an attitude that helped mold the young man’s worldview.

[…]

He turned cynical, viewing government as arrogant and inefficient.

Mercer earned a PhD in computer science from the University of Illinois, then joined IBM and its speech-recognition group in 1972.

***

Peter Brown’s father Henry Brown introduced the world’s first money-market mutual fund.  Few investors showed any interest, however.  Henry worked every day except Christmas in 1972, when Peter was seventeen.  Zuckerman:

His lucky break came the next year in the form of a New York Times article about the fledgling fund.  Clients began calling, and soon Henry and his partner were managing $100 million in their Reserve Primary Fund.  The fund grew, reaching billions of dollars, but Henry resigned, in 1985, to move [with his wife] to the Brown family’s farm in a Virginia hamlet, where he raised cattle on five hundred acres.

Zuckerman continues:

Peter reserved his own ambitions for science and math.  After graduating from Harvard University with an undergraduate degree in mathematics, Brown joined a unit of Exxon that was developing ways to translate spoken language into computer text, an early form of speech-recognition technology.  Later, he’d earn a PhD in computer science from Carnegie Mellon University in Pittsburgh.

In 1984, Brown joined IBM’s speech-recognition group.  Zuckerman:

Brown, Mercer, and their fellow mathematicians and scientists, including the group’s hard-driving leader, Fred Jelinek, viewed language very differently from the traditionalists.  To them, language could be modeled like a game of chance.  At any point in a sentence, there exists a certain probability of what might come next, which can be estimated based on past, common usage…

Photo by Alexander Bogdanovich

Their goal was to feed their computers with enough data of recorded speech and written text to develop a probabilistic, statistical model capable of predicting likely word sequences based on sequences of sounds…

In mathematical terms, Brown, Mercer, and the rest of Jelinek’s team viewed sounds as the output of a sequence in which each step along the way is random, yet dependent on the previous step—a hidden Markov model.  A speech-recognition system’s job was to take a set of observed sounds, crunch the probabilities, and make the best possible guess about the “hidden” sequences of words that could have generated those sounds.  To do that, the IBM researchers employed the Baum-Welch algorithm—codeveloped by Jim Simons’s early trading partner Lenny Baum—to zero in on the various language probabilities.  Rather than manually programming in static knowledge about how language worked, they created a program that learned from data.

Mercer jumped rope to stay in shape.  He had a hyper-efficient style of communication, usually saying nothing and otherwise saying only a few words.  Brown, by contrast, was more approachable and animated.

Jelinek created a ruthless and fierce culture.  Zuckerman:

Researchers would posit ideas and colleagues would do everything they could to eviscerate them, throwing personal jabs along the way.  They’d fight it out until reaching a consensus on the merits of the suggestion… It was no-holds-barred intellectual combat.

…Brown stood out for having unusual commercial instincts, perhaps the result of his father’s influence.  Brown urged IBM to use the team’s advances to sell new products to customers, such as a credit-evaluation service, and even tried to get management to let them manage a few billion dollars of IBM’s pension-fund investments with their statistical approach, but failed to garner much support.

[…]

At one point, Brown learned of a team of computer scientists, led by a former Carnegie Mellon classmate, that was programming a computer to play chess.  He set out to convince IBM to hire the team.  One winter day, while Brown was in an IBM men’s room, he got to talking with Abe Peled, a senior IBM research executive, about the exorbitant cost of the upcoming Super Bowl’s television commercials.  Brown said he had a way to get the company exposure at a much lower cost—hire the Carnegie Mellon team and reap the resulting publicity when their machine beat a world champion in chess…

The IBM brass loved the idea and hired the team, which brought its Deep Thought program along.

Mercer was skeptical that hedge funds create good for society.  But he agreed to visit Renaissance and he was impressed that the company seemed to be about science.  Also, both Mercer and Brown were not paid much at IBM.  Brown, for his part, became more interested when he learned that Simons’s had worked with Lenny Baum, coinventor of the Baum-Welch algorithm.

Simons offered to double their salaries.  Mercer and Brown joined Renaissance in 1993.

 

CHAPTER TEN

Brown suggested that Renaissance interview David Magerman, whom Brown knew from IBM.  Magerman’s specialty was programming, which Renaissance needed.  Soon Renaissance hired Magerman.

Magerman had a difficult upbringing, growing up without much money.  His father was a math whiz who’d never been able to develop his talents.  He took it out on his son.  Magerman was left with a desire to earn praise from people in power, some of whom Magerman saw as father figures.  He also seemed to pick fights unnecessarily.  Zuckerman:

“I needed to right wrongs and fight for justice, even if I was turning molehills into mountains,” Magerman acknowledges.  “I clearly had a messiah complex.”

Magerman studied mathematics and computer science at the University of Pennsylvania.  He excelled.  Zuckerman:

At Stanford University, Magerman’s doctoral thesis tackled the exact topic Brown, Mercer, and other IBM researchers were struggling with: how computers could analyze and translate language using statistics and probability.

Although Simons had initially had Brown and Mercer working in different areas of Renaissance, the pair had been working together in their spare time on how to fix Renaissance’s stock-trading system.  Soon Mercer figured out the key problem.  Simons let Mercer join Brown in the stock-research area.  Zuckerman:

The Brown-Mercer reunion represented a new chapter in an unusual partnership between two scientists with distinct personalities who worked remarkably well together.  Brown was blunt, argumentative, persistent, loud, and full of energy.  Mercer conserved his words and rarely betrayed emotion, as if he was playing a never-ending game of poker.  The pair worked, though, yin with yang.

[…]

…[While at IBM, they] developed a certain work style—Brown would quickly write drafts of their research and then pass them to Mercer, a much better writer, who would begin slow and deliberate rewrites.

Brown and Mercer threw themselves into their new assignment [to revamp Renaissance’s stock-trading model.]  They worked late into the evening and even went home together; during the week they shared a living space in the attic of a local elderly woman’s home, returning to their families on weekends.  Over time, Brown and Mercer discovered methods to improve Simons’s stock-trading system.

Zuckerman:

[Brown and Mercer] decided to program the necessary limitations and qualifications into a single trading system that could automatically handle all potential complications.  Since Brown and Mercer were computer scientists, and they had spent years developing large-scale software projects at IBM and elsewhere, they had the coding chops to build a single automated system for trading stocks.

[…]

Brown and Mercer treated their challenge as a math problem, just as they had with language recognition at IBM.  Their inputs were the fund’s trading costs, its various leverages, risk parameters, and assorted other limitations and requirements.  Given all those factors, they built the system to solve and construct an ideal portfolio, making optimal decisions, all day long, to maximize returns.

Zuckerman writes:

The beauty of the approach was that, by combining all their trading signals and portfolio requirements into a single, monolithic model, Renaissance could easily test and add new signals, instantly knowing if the gains from a potential new strategy were likely to top its costs.  They also made their system adaptive, or capable of learning and adjusting on its own…

Photo by Weerapat Wattanapichayakul

If the model’s recommended trades weren’t executed, for whatever reason, it self-corrected, automatically searching for buy-or-sell orders to nudge the portfolio back where it needed to be… The system repeated on a loop several times an hour, conducting an optimization process that weighed thousands of potential trades before issuing electronic trade instructions.  Rivals didn’t have self-improving models; Renaissance now had a secret weapon, one that would prove crucial to the fund’s future success.

Eventually, Brown and Mercer developed an elaborate stock-trading system that featured a half million lines of code, compared to tens of thousands of lines in [the] old system.  The new system incorporated all necessary restrictions and requirements; in many ways, it was just that kind of automated trading system Simons had dreamed of years earlier.  Because [the] fund’s stock trades were now less sensitive to the market’s fluctuations, it began holding on to shares a bit longer, two days or so, on average.

…[The model] continued to identify enough winning trades to make serious money, usually by wagering on reversions after stocks got out of whack.  Over the years, Renaissance would add twists to this bedrock strategy, but, for more than a decade, those would just be second order complements to the firm’s core reversion-to-the-mean predictive signals.

However, there were problems early on.  Zuckerman:

It soon became clear that the new stock-trading system couldn’t handle much money, undermining Simons’s original purpose in pushing into equities.  Renaissance placed a puny $35 million in stocks; when more money was traded, the gains dissipated… Even worse, Brown and Mercer couldn’t figure out why their system was running into so many problems.

Brown and Mercer began putting together the old team from IBM, including Magerman.

Simons gave Brown and Mercer six months to get the fund’s trading system working.

Magerman started spending all his free time trying to fix the trading system.  He basically lived at the office.  Eventually he identified two errors and was able to fix them.  Magerman told Brown, who didn’t seem too excited.  Magerman then showed Mercer, who had written all the code.  Mercer checked Magerman’s work and then told him that he was right.

Brown and Mercer restarted the system and profits immediately started coming in.

 

CHAPTER ELEVEN

Renaissance’s trading system gained 21 percent in 1997, lower than the 32, 38, and 71 percent gained in 1996, 1995, and 1994.  The system still had issues.

Simons drew on his experiences with the IDA and also from when he managed talented mathematicians at Stony Brook.  One lesson Simons had learned was the importance of having researchers work together.  Zuckerman:

…Medallion would have a single, monolithic trading system.  All staffers enjoyed full access to each line of the source code underpinning their moneymaking algorithms, all of it readable in cleartext on the firm’s internal network.  There would be no corners of the code accessible only to top executives; anyone could make experimental modifications to improve the trading system.  Simons hoped his researchers would swap ideas, rather than embrace private projects…

Simons created a culture of unusual openness.  Staffers wandered into colleagues’ offices offering suggestions and initiating collaborations.  When they ran into frustrations, the scientists tended to share their work and ask for help, rather than move on to new projects, ensuring the promising ideas weren’t “wasted,” as Simons put it.  Groups met regularly, discussing intimate details of their progress and fielding probing questions from Simons… Once a year, Simons paid to bring employees and their spouses to exotic vacation locales, strengthening the camaraderie.

Furthermore, peer pressure was used.  Employees were often working on presentations, and felt the need to try to impress one another.

Simons gave all employees the chance to share in Renaissance’s profits based on clear and transparent formulas.  Zuckerman notes:

Simons began sharing equity, handing a 10 percent stake in the firm to Laufer and, later, giving sizable slices to Brown, Mercer, and Mark Silber, who was now the firm’s chief financial officer, and others, steps that reduced Simons’s ownership to just over 50 percent.  Other top-performing employees could buy shares, which represented equity in the firm.  Staffers also could invest in Medallion, perhaps the biggest perk of them all.

Hiring wasn’t easy.  To attract scientists and mathematicians, Renaissance employees would emphasize certain positive parts of their job.  Solving puzzles was fun.  There was camaraderie.  And things moved at a fast pace.  Zuckerman:

“You have money in the bank or not, at the end of the day,” Mercer told science writer Sharon McGrayne.  “You don’t have to wonder if you succeeded… it’s just a very satisfying thing.”

Zuckerman continues:

By 1997, Medallion’s staffers had settled on a three-step process to discover statistically significant moneymaking strategies, or what they called their trading signals.  Identify anomalous patterns in historic pricing data; make sure the anomalies were statistically significant, consistent over time, and nonrandom; and see if the identified pricing behavior could be explained in a reasonable way.

[…]

…more than half of the trading signals Simons’s team was discovering were nonintuitive, or those they couldn’t fully understand.  Most quant firms ignore signals if they can’t develop a reasonable hypothesis to explain them, but Simons and his colleagues never liked spending too much time searching for the causes of market phenomena.  If their signals met various measures of statistical strength, they were comfortable wagering on them.  They only steered clear of the most preposterous ideas.

Zuckerman adds:

It’s not that they wanted trades that didn’t make any sense; it’s just that these were the statistically valid strategies they were finding.  Recurring patterns without apparent logic to explain them had an added bonus: They were less likely to be discovered and adopted by rivals, most of whom wouldn’t touch these kinds of trades.

The danger was that unexplainable signals could be simple coincidences.  Zuckerman:

Often, the Renaissance researchers’ solution was to place such head-scratching signals in their trading system, but to limit the money allocated to them, at least at first, as they worked to develop an understanding of why the anomalies appeared.  Over time, they frequently discovered reasonable explanations, giving Medallion a leg up on firms that had dismissed the phenomena.  They ultimately settled on a mix of sensible signals, surprising trades with strong statistical results, and a few bizarre signals so reliable they couldn’t be ignored.

Zuckerman again:

By then, Medallion increasingly was relying on strategies that its system taught itself, a form of machine learning.  The computers, fed with enough data, were trained to spit out their own answers.  A consistent winner, for example, might automatically receive more cash, without anyone approving the shift or even being aware of it.

Photo by SJMPhotos

Simons developed enough enthusiasm and confidence in the future of Renaissance that he moved the firm to a new office.  The compound was wood and glass.  Each office had a view of the woods.  There was a gym, lighted tennis courts, a library with a fireplace, and a large auditorium where Simons hosted biweekly seminars from visiting scholars (usually having little to do with finance).  The cafeteria and common areas were large, with plenty of whiteboards, so that staffers could meet, discuss, and debate.

Zuckerman describes Brown and Mercer:

Intense and energetic, Brown hustled from meeting to meeting, riding a unicycle through the halls and almost running over colleagues.  Brown worked much of the night on a computer near the Murphy bed in his office, grabbing a nap when he tired.

[…]

Analytical and unemotional, Mercer was a natural sedative for his jittery partner.  Mercer worked hard, but he liked to go home around six p.m.

One important factor of Renaissance’s success, as Zuckerman explains:

If a strategy wasn’t working, or when market volatility surged, Renaissance’s system tended to automatically reduce positions and risk.

This contrasted with many other hedge funds, including Long-Term Capital Management (LTCM), which tended to hold or to increase positions that had gone against them in the belief that they would ultimately be proved right.  In the summer of 1998, LTCM blew up when its positions went against the fund.  Traders at LTCM were more right than wrong, but because they had a great deal of leverage and because they maintained or increased their bets, they simply didn’t have enough capital to survive.  Renaissance wouldn’t make this mistake because, as noted, the fund typically reduced positions and risk if a strategy wasn’t working or if market volatility picked up.

Despite all the improvements, Renaissance’s stock-trading system still only generated about 10 percent of the fund’s profits.

In March 2000, Medallion encountered major difficulties.  Zuckerman:

It wasn’t just the mounting losses that had everyone concerned—it was the uncertainty over why things were so bad.  The Medallion portfolio held commodities, currencies, and bond futures, and its stock portfolio was largely composed of offsetting positions aimed at sidestepping broad market moves.  The losses shouldn’t be happening.  But because so many of the system’s trading signals had developed on their own through a form of machine learning, it was hard to pinpoint the exact cause of the problems or when they might ebb; the machines seemed out of control.

Brown was freaking out and barely sleeping.  Magerman felt nauseous.  Mercer tried to keep his composure.  But many were very worried.

Simons urged his team to stick with the model.  After more all-nighters, a few researchers thought they figured out the problem.  If certain stocks rallied in the preceding weeks, Medallion’s system would buy them on the expectation that the rally would continue.  But as the market sold off, this momentum strategy wasn’t working at all.  So the team put a halt to the fund’s momentum strategy.  Immediately, Medallion began making money again.  Zuckerman comments:

By the fall of 2000, word of Medallion’s success was starting to leak out.  That year, Medallion soared 99 percent, even after it charged clients 20 percent of their gains and 5 percent of the money invested with Simons.  Over the previous decade, Medallion and its 140 employees had enjoyed a better performance than funds managed by George Soros, Julian Robertson, Paul Tudor Jones, and other investing giants.  Just as impressive, Medallion had recorded a Sharpe ratio of 2.5 in its most recent five-year period, suggesting the fund’s gains came with low volatility and risk compared with those of many competitors.

 

Part Two: Money Changes Everything

CHAPTER TWELVE

Zuckerman writes:

Something unusual was going on at Jim Simons’s hedge fund in 2001.

Profits were piling up as Renaissance began digesting new kinds of information.  The team collected every trade order, including those that hadn’t been completed, along with annual and quarterly earnings reports, records of stock trades by corporate executives, government reports, and economic predictions and papers.

[…]

Soon, researchers were tracking newspaper and newswire stories, internet posts, and more obscure data—such as offshore insurance claims—racing to get their hands on pretty much any information that could be quantified and scrutinized for its predictive value.  The Medallion fund became something of a data sponge, soaking up a terabyte, or one trillion bytes, of information annually, buying expensive disc drives and processors to digest, store, and analyze it all, looking for reliable patterns.

“There’s no data like more data,” Mercer told a colleague, an expression that became the firm’s hokey mantra.

Renaissance’s goal was to predict the price of a stock or other investment “at every point in the future,” Mercer later explained.  “We want to know in three seconds, three days, three weeks, and three months.”

Zuckerman again:

It became clear to Mercer and others that trading stocks bore similarities to speech recognition, which was part of why Renaissance continued to raid IBM’s computational linguistics team.  In both endeavors, the goal was to create a model capable of digesting uncertain jumbles of information and generating reliable guesses about what might come next—while ignoring traditionalists who employed analysis that wasn’t nearly as data driven.

[…]

…After soaring 98.5 percent in 2000, the Medallion fund rose 33 percent in 2001.  By comparison, the S&P 500, the commonly used barometer of the stock market, managed a measly average gain of 0.2 percent over those two years, while rival hedge funds gained 7.3 percent.

Zuckerman adds:

Investment professionals generally judge a portfolio’s risk by its Sharpe ratio, which measures returns in relation to volatility; the higher one’s Sharpe, the better.  For most of the 1990s, Medallion had a strong Sharpe ratio of about 2.0, double the level of the S&P 500.  But adding foreign-market algorithms and improving Medallion’s trading techniques sent its Sharpe soaring to about 6.0 in early 2003, about twice the ratio of the largest quant firms and a figure suggesting there was nearly no risk of the fund losing money over a whole year.

(Holy grail, Photo by Charon)

Simons’s team appeared to have discovered something of a holy grail in investing: enormous returns from a diversified portfolio generating relatively little volatility and correlation to the overall market.  In the past, a few others had developed investment vehicles with similar characteristics.  They usually had puny portfolios, however.  No one had achieved what Simons and his team had—a portfolio as big as $5 billion delivering this kind of astonishing performance.

Zuckerman later says:

Brown and Mercer’s staffers often spent the night programming their computers, competing to see who could stay in the office the longest, then rushing back in the morning to see how effective their changes had been.  If Brown was going to push himself all day and sleep by his computer keyboard at night, his underlings felt the need to keep up…

[…]

By 2003, the profits of Brown and Mercer’s stock-trading group were twice those of Laufer’s futures team, a remarkable shift in just a few years.  Rewarding his ascending stars, Simons announced that Brown and Mercer would become executive vice presidents of the entire firm, co-managing all of Renaissance’s trading, research, and technical activities.

Zuckerman:

In 2002, Simons increased Medallion’s investor fees to 36 percent of each year’s profits, raising hackles among some clients.  A bit later, the firm boosted the fees to 44 percent.  Then, in early 2003, Simons began kicking all his investors out of the fund.  Simons had worried that performance would ebb if Medallion grew too big, and he preferred that he and his employees kept all the gains.  But some investors had stuck with Medallion through difficult periods and were crushed.

Nick Simons, Jim Simons’s third-eldest son shared his father’s passions for hiking and adventure.  He was taking a trip around the world before he was going to learn organic chemistry and apply for medical school.  A week before he was scheduled to come home, he went freediving near Amed, a coastal strip of fishing villages in eastern Bali.  While freediving, one person would be down, and the other up, in order to keep track of one another.  Nick and a friend thus took turns.  At one point, Nick’s friend had to go ashore to unfog his mask.  When he got back out, he found Nick’s body near the bottom.  They weren’t able to resuscitate him.  Nick had drowned.

When Jim and Marilyn found out, they were inconsolable.  Zuckerman:

In September, Jim, Marilyn, and other family members traveled to Nepal for the first time, joining some of Nick’s friends in searching for a way to continue Nick’s legacy.  Nick had been drawn to Kathmandu and had an interest in medicine, so they funded a maternity ward at a hospital in the city.  Later, Jim and Marilyn would start the Nick Simons Institute, which offers healthcare assistance to those living in Nepal’s rural areas, most of whom don’t have basic emergency services.

 

CHAPTER THIRTEEN

Medallion paid out its profits to investors—mostly its own employees by now—once per year.  That helped keep the fund small enough to maintain the same high level of profitability.  Simons, Henry Laufer, and others were sure that if the fund got much bigger, its profitability would suffer.

As a result, there were some profitable investments—mostly of a longer-term nature—that the fund couldn’t implement.  So Simons thought that they should launch a second fund.  This fund couldn’t be as profitable as Medallion; however, it could manage a lot more money than Medallion, giving outside investors a chance to invest with Renaissance.

One reason Simons wanted to launch the new fund was to give his scientists and mathematicians a new challenge.  Many of them now had more money than they ever imagined, so keeping them challenged and energized was something Simons worried about.

Photo by Feng Yu

Zuckerman:

His researchers settled on one that would trade with little human intervention, like Medallion, yet would hold investments a month or even longer.  It would incorporate some of Renaissance’s usual tactics, such as finding correlations and patterns in prices, but would add other, more fundamental strategies, including buying inexpensive shares based on price-earnings ratios, balance-sheet data, and other information.

After thorough testing, the scientists determined the new hedge fund could beat the stock market by a few percentage points each year, while generating lower volatility than the overall market.  It could produce the kinds of steady returns that hold special appeal for pension funds and other large institutions.  Even better, the prospective fund could score those returns even if it managed as much as $100 billion, they calculated, an amount that would make it history’s largest hedge fund.

The new fund was called Renaissance Institutional Equities Fund, or RIEF.  The firm emphasized to investors that the new fund wouldn’t resemble Medallion.  Yet many investors ignored the warning, thinking it was the same scientists and the same research.  Soon RIEF had $14 billion under management.  Zuckerman:

Other than making the occasional slipup, Simons was an effective salesman, a world-class mathematician with a rare ability to connect with those who couldn’t do stochastic differential equations.  Simons told entertaining stories, had a dry sense of humor, and held interests far afield from science and moneymaking.  He also demonstrated unusual loyalty and concern for others, qualities the investors may have sensed.

Zuckerman adds:

By the spring of 2007, it was getting hard to keep investors away.  Thirty-five billion dollars had been ploughed into RIEF, making it one of the world’s largest hedge funds… Simons made plans for other new funds, initiating work on the Renaissance Institutional Futures Fund, RIFF, to trade futures contracts on bonds, currencies, and other assets in a long-term style.  A new batch of scientists was hired, while staffers from other parts of the company lent a hand, fulfilling Simons’s goal of energizing and unifying staffers.

During the summer of 2007, a few quant funds were in trouble.  There was selling that seemed to be impacting all the quant funds, despite the fact that these funds thought their investment strategy was unique.  The downturn became known as the “quant quake.”

Medallion was approaching the point where it could face a margin call.  During the technology-stock meltdown in 2000, Brown hadn’t known what to do.  But this time, he knew: Medallion should maintain, and maybe even increase, its bets.  Mercer agreed with Brown.  So did Henry Laufer, who said: “Trust the models—let them run.”  Simons, however, disagreed.  Zuckerman:

Simons shook his head.  He didn’t know if his firm could survive much more pain.  He was scared.  If losses grew, and they couldn’t come up with enough collateral, the banks would sell Medallion’s positions and suffer their own huge losses.  If that happened, no one would deal with Simons’s fund again.  It would be a likely death blow, even if Renaissance suffered smaller financial losses than its bank lenders.

Medallion needed to sell, not buy, he told his colleagues.

“Our job is to survive,” Simons said.  “If we’re wrong, we can always add [positions] later.”

Brown seemed shocked by what he was hearing.  He had absolute faith in the algorithms he and his fellow scientists had developed…

Medallion began reducing its positions.  Zuckerman again:

Some rank-and-file senior scientists were upset—not so much by the losses, but because Simons had interfered with the trading system and reduced positions…

“You’re dead wrong,” a senior researcher emailed Simons.

“You believe in the system, or you don’t,” another scientist said, with some disgust.

But Simons continued to oversee a reduction in Medallion’s positions.  He wondered aloud how far the selloff could go, and he wanted to make sure Medallion survived.

Soon Simons ordered his firm to stop selling.  Their positions seemed to be stabilizing.  Some Renaissance scientists complained that their gains would have been larger if they had maintained their positions rather than lightening up.  But Simons claimed that he would make the same decision again.  Remarkably, by the end of 2007, Medallion had gained 86 percent.

***

Zuckerman writes about 2008:

…[The] Medallion fund thrived in the chaos, soaring 82 percent that year, helping Simons make over $2 billion in personal profits.  The enormous gains sparked a call from a House of Representatives committee asking Simons to testify as part of its investigation into the causes of the financial collapse.  Simons prepped diligently with his public-relations advisor Jonathan Gasthalter.  With fellow hedge-fund managers George Soros to his right and John Paulson on his left, Simons told Congress that he would back a push to force hedge funds to share information with regulators and that he supported higher taxes for hedge-fund managers.

Simons was something of an afterthought, however, both at the hearings and in the finance industry itself.  All eyes were on Paulson, Soros, and a few other investors who, unlike Simons, had successfully anticipated the financial meltdown.  They did it with old-fashioned investment research, a reminder of the enduring potential and appeal of those traditional methods.

Medallion was now managing $10 billion, and had averaged 45 percent returns—net of fees—since 1988.  Simons had created the best long-term track record in the world.  Now, at age seventy-two, Simons decided to turn the reins over to Brown and Mercer.

 

CHAPTER FOURTEEN

Zuckerman:

Jim Simons liked making money.  He enjoyed spending it, too.

Stepping down from Renaissance gave Simons—who, by then, was worth about $11 billion—more time on his 220-foot yacht, Archimedes.  Named for the Greek mathematician and inventor, the $100 million vessel featured a formal dining room that sat twenty, a wood-burning fireplace, a spacious Jacuzzi, and a grand piano.  Sometimes, Simons flew friends on his Gulfstream G450 to a foreign location, where they’d join Jim and Marilyn on the super-yacht.

Zuckerman continues:

Years earlier, Marilyn had carved out space in her dressing room to launch a family foundation.  Over time, she and Jim gave over $300 million to Stony Brook University, among other institutions.  As Simons edged away from Renaissance, he became more personally involved in their philanthropy.  More than anything, Simons relished tackling big problems.  Soon, he was working with Marilyn to target two areas in dire need of solutions: autism research and mathematics education.

In 2004, Simons helped launch Math for America, a nonprofit focused on math education and supporting outstanding teachers with annual stipends of $15,000.

Simons remained Renaissance’s chairman.  At first, the transition was difficult.  But soon Simons found his philanthropic goals to be challenging and engaging.  Zuckerman:

…[Renaissance] now employed about 250 staffers and over sixty PhDs, including experts in artificial intelligence, quantum physicists, computational linguists, statisticians, and number theorists, as well as other scientists and mathematicians.

Astronomers, who are accustomed to scrutinizing large, confusing data sets and discovering evidence of subtle phenomena, proved especially capable of identifying overlooked market patterns…

Medallion still did bond, commodity, and currency trades, and it made money from trending and reversion-predicting signals… More than ever, though, it was powered by complex equity trades featuring a mix of complex signals, rather than simple pair trades…

Zuckerman adds:

The gains on each trade were never huge, and the fund only got it right a bit more than half the time, but that was more than enough.

“We’re right 50.75 percent of the time… but we’re 100 percent right 50.75 percent of the time,” Mercer told a friend.  “You can make billions that way.”

Zuckerman continues:

Driving these reliable gains was a key insight: Stocks and other investments are influenced by more factors and forces than even the most sophisticated investors appreciated…

[…]

…By analyzing and estimating hundreds of financial metrics, social media feeds, barometers of online traffic, and pretty much anything that can be quantified and tested, they uncovered new factors, some borderline impossible for most to appreciate.

“The inefficiencies are so complex they are, in a sense, hidden in the markets in code,” a staffer says.  “RenTec decrypts them.  We find them across time, across risk factors, across sectors and industries.”

Even more important: Renaissance concluded that there are reliable mathematical relationships between all these forces.  Applying data science, the researchers achieved a better sense of when various factors were relevant, how they interrelated, and the frequency with which they influenced shares.  They also tested and teased out subtle, nuanced mathematical relationships between various shares—what staffers call multidimensional anomalies—that other investors were oblivious to or didn’t fully understand.

Photo by Funtap P

Zuckerman quotes a Renaissance employee:

“There is no individual bet we make that we can explain by saying we think one stock is going to go up or another down,” a senior staffer says.  “Every bet is a function of all the other bets, our risk profile, and what we expect to do in the near and distant future.  It’s a big, complex optimization based on the premise that we predict the future well enough to make money from our predictions, and that we understand risk, cost, impact, and market structure well enough to leverage the hell out of it.”

Renaissance also excelled at disguising its buying and selling so that others wouldn’t know what it was doing.  This helped to minimize the costs of trading.

***

Bob Mercer, while being highly intelligent, surprised his colleagues with political views that didn’t seem to be based on much evidence or critical thinking.  Of course, part of this could be that Mercer’s colleagues tended to have a liberal bias.  In any case, Mercer was a Rational Rifle Association member who believed in guns and gold.  Mercer didn’t like taxes and he was a skeptic of climate change.

Furthermore, Mercer emerged as a key donor to right-wing causes.  Unusually, while most big contributors wanted something in return from politicians, Mercer didn’t ask for anything in return.  Zuckerman:

Mercer’s penchant for privacy limited his activity, however, as did his focus on Renaissance.  It was his second-oldest daughter, Rebekah, who started showing up at conservative fund-raising events and other get-togethers, becoming the family’s public face, and the one driving its political strategy.

Zuckerman continues:

…in 2011, the Mercers met conservative firebrand Andrew Breitbart at a conference.  Almost immediately, they were intrigued with his far-right news organization, Breitbart News Network, expressing interest in funding its operations.  Breitbart introduced the Mercers to his friend, Steve Bannon, a former Goldman Sachs banker, who drew up a term sheet under which the Mercer family purchased nearly 50 percent of Breitbart News for $10 million.

In March 2012, Breitbart collapsed on a Los Angeles sidewalk and died of heart failure at the age of forty-three.  Bannon and the Mercers convened an emergency meeting in New York to determine the network’s future, and decided that Bannon would become the site’s executive chairman.  Over time, the site became popular with the “alt-right,” a loose conglomeration of groups, some of which embraced tenets of white supremacy and viewed immigration and multiculturalism as threats.

[…]

Bannon helped broker a deal for Mercer to invest in an analytics firm called Cambridge Analytica, the US arm of the British behavioral research company SCL Group.  Cambridge Analytica specialized in the kinds of advanced data Mercer was accustomed to parsing at Renaissance, and the type of information that Rebekah said the GOP lacked.

Zuckerman writes:

In February 2014, Mercer and other conservative political donors gathered at New York’s Pierre hotel to strategize about the 2016 presidential election.  He told attendees he had seen data indicating that mainstream Republicans, such as Jeb Bush and Marco Rubio, would have difficulty winning.  Only a true outsider with a sense of the voters’ frustrations could emerge victorious, Mercer argued.

At a gathering of GOP backers who were going to meet Trump, Rebekah walked straight to Trump.  Zuckerman:

“It’s bad,” Trump acknowledged.

“No, it’s not bad—it’s over,” she told Trump.  “Unless you make a change.”

Rebekah told Trump to bring in Steve Bannon and Kellyanne Conway.  Zuckerman:

Before long, Bannon was running the campaign, and Conway was its manager, becoming a ubiquitous and effective television presence.  Bannon helped instill order on the campaign, making sure Trump focused on two things—disparaging Clinton’s character and promoting a form of nationalism that Bannon branded “America First” …

***

Meanwhile, Jim Simons was supporting Democrats.  Zuckerman says:

Ever since he and his childhood friend, Jim Harpel, had driven across the country and witnessed some of the hardships experienced by minorities and others, Simons had leaned left politically…. By the middle of 2016, Simons had emerged as the most important supporter of the Democratic Party’s Priorities USA Action super PAC and a key backer of Democratic House and Senate candidates.  By the end of that year, Simons would donate more than $27 million to Democratic causes.  Marilyn Simons was even more liberal than her husband, and Jim’s son, Nathaniel, had established a nonprofit foundation focused on climate change mitigation and clean-energy policy, issues the Trump campaign generally mocked or ignored.

Associates and others at Renaissance began asking Jim Simons if he could do anything about Mercer’s political activities.  But Mercer had always been pleasant and respectful to Simons.  Zuckerman:

“He’s a nice guy,” he insisted to a friend.  “He’s allowed to use his money as he wishes.  What can I do?”

 

CHAPTER FIFTEEN

Zuckerman writes:

Rebekah Mercer was emerging as a public figure in her own right… GQ named Mercer the seventeenth most powerful person in Washington, DC, calling her “the First Lady of the alt-right.”  The family’s political clout, along with its ongoing support for the president-elect, seemed assured.

David Magerman was unhappy with Bob Mercer’s support of Trump.  Magerman thought that much of what Trump was doing was harming the country.  Eventually, he did an interview with Gregory Zuckerman of The Wall Street Journal.  Magerman didn’t hold back.  Soon after that, he got a call from Renaissance telling him he was suspended without pay.

The Mercers were facing other consequences of their political activities.  Zuckerman:

At one point, the New York State Democratic Committee ran a television advertisement flashing Bob and Rebekah Mercer’s faces on the screen, saying they were the “same people who bankrolled Trump’s social media bot army and Steve Bannon’s extremist Breitbart News.”

In March 2017, about sixty demonstrators gathered outside Mercer’s home, decrying his funding of far-right causes and calling for higher taxes on the wealthy.  A week later, a second group held a protest, some holding signs reading: “Mercer Pay Your Taxes.”

[…]

The Mercers received death threats, friends said, forcing the family to hire security.  For a family that relished its privacy, their growing infamy was both shocking and disturbing.

Magerman felt some uncertainty about whether his public criticism of his boss was the right move.  Zuckerman:

Magerman had mixed feelings of his own.  He had made so much money at the firm that he didn’t have to worry about the financial pain of getting fired.  He loathed what Mercer was doing to the country and wanted to stop his political activity.  But Magerman also remembered how kind Mercer and his wife had been to him when he first joined the firm, inviting him to dinners at Friendly’s and movie nights with their family.  Magerman respected Bob for his intelligence and creativity, and a big part of him still yearned to please the powerful men in his life.  At that point, Magerman had spent two decades at Renaissance and he felt an appreciation for the firm.  He decided that if he could go on speaking about Mercer’s politics, he’d return to his old job.

Magerman decided to attend a poker tournament at New York’s St. Regis hotel benefitting Math for America.

Photo by Ali Altug Kirisoglu

He wanted to reintroduce himself to key Renaissance people.  Zuckerman:

The event was a highly anticipated annual showdown for quants, professional poker players, and others.  Magerman knew Simons, Mercer, Brown, and other Renaissance executives would be there.

Rebekah Mercer was there, too.  Magerman tried to approach her, and she started yelling at him.  Zuckerman:

“How could you do this to my father?  He was so good to you,” she said.

[…]

“You’re pond scum,” Mercer told him, repeatedly.  “You’ve been pond scum for twenty-five years.  I’ve always known it.”

Get out of here, she told Magerman.

Soon security forced Magerman to leave.  Shortly thereafter, Renaissance fired Magerman.

Meanwhile, criticism of Renaissance grew, focused on Mercer.  Zuckerman:

…nearly fifty protesters picketed the hedge fund itself, saying Mercer was their target, adding to the discomfort of executives, who weren’t accustomed to such negative publicity.

By October 2017, Simons was worried the controversy was jeopardizing Renaissance’s future.  The firm’s morale was deteriorating.

Finally, Simons made the difficult decision to ask Mercer to step down from his co-CEO role.

Overall, while the Mercers pulled back on many activities, they seemed to be happy with Trump.  Zuckerman:

The Mercers told friends they were happy the Trump administration had cut taxes and chosen conservative judges, among other moves, suggesting they didn’t regret becoming so involved in national politics.

 

CHAPTER SIXTEEN

In late December 2018, the stock market was collapsing.  The S&P 500 Index fell almost 10 percent during that month, the worst December since 1931.  Simons was worried.  He called Ashvin Chhabra, who runs Euclidean Capital, a firm that manages Simons’s money.  Simons asked Chhabra whether they should sell short.  Chhabra suggested they wait until the market calmed down before considering whether to do anything.  Simons agreed.

Zuckerman comments:

Hanging up, neither Simons nor Chhabra focused on the rich irony of their exchange.  Simons had spent more than three decades pioneering and perfecting a new way to invest.  He had inspired a revolution in the financial world, legitimizing a quantitative approach to trading.  By then, it seemed everyone in the finance business was trying to invest the Renaissance way: digesting data, building mathematical models to anticipate the direction of various investments, and employing automated trading systems.  The establishment had thrown in the towel.  Today, even banking giant JP Morgan Chase puts hundreds of its new investment bankers and investment professionals through mandatory coding lessons.  Simons’s success had validated the field of quantitative investing.

[…]

The goal of quants like Simons was to avoid relying on emotions and gut instinct.  Yet, that’s exactly what Simons was doing after a few difficult weeks in the market.  It was a bit like Oakland A’s executive Billy Beane scrapping his statistics to draft a player with the clear look of a star.

Zuckerman continues:

Simons’s phone call is a stark reminder of how difficult it can be to turn decision-making over to computers, algorithms, and models—even, at times, for the inventors of these very approaches.  His conversation with Chhabra helps explain the faith investors have long placed in stock-and-bond pickers dependent on judgment, experience, and old-fashioned research.

By 2019, however, confidence in the traditional approach had waned.  Years of poor performance had investors fleeing actively managed stock-mutual funds, or those professing an ability to beat the market’s returns.  At that point, these funds, most of which embrace traditional approaches to investing, controlled just half of the money entrusted by clients in stock-mutual funds, down from 75 percent a decade earlier.  The other half of the money was in index funds and other so-called passive vehicles, which simply aim to match the market’s returns, acknowledging how challenging it is to top the market.

Zuckerman later notes:

There are reasons to think the advantages that firms like Renaissance enjoy will only expand amid an explosion of new kinds of data that their computer-trading models can digest and parse.  IBM has estimated that 90 percent of the world’s data sets have been created in the last two years alone, and that forty zettabytes—or forty-four trillion gigabytes—of data will be created by 2020, a three-hundred-fold increase from 2005.

Today, almost every kind of information is digitized and made available as part of huge data sets, the kinds that investors once only dreamed of tapping.  The rage among investors is for alternative data, which includes just about everything imaginable, including instant information from sensors and satellite images around the world.  Creative investors test for money-making correlations and patterns by scrutinizing the tones of executives on conference calls, traffic in the parking lots of retail stores, records of auto-insurance applications, and recommendations by social media influencers.

Rather than wait for figures on agricultural production, quants examine sales of farm equipment or satellite images of crop yields.  Bills of lading for cargo containers can give a sense of global shifts.  Systematic traders can even get cell phone-generated data on which aisles, and even which shelfs, consumers are pausing to browse within stores.  If you seek a sense of the popularity of a new product, Amazon reviews can be scraped.  Algorithms are being developed to analyze the backgrounds of commissioners and others at the Food and Drug Administration to predict the likelihood of a new drug’s approval.

Zuckerman adds:

Years after Simons’s team at Renaissance adopted machine-learning techniques, other quants have begun to embrace these methods.  Renaissance anticipated a transformation in decision-making that’s sweeping almost every business and walk of life.  More companies and individuals are accepting and embracing models that continuously learn from their successes and failures.  As investor Matthew Granade has noted, Amazon, Tencent, Netflix, and others that rely on dynamic, ever-changing models are emerging dominant.  The more data that’s fed to the machines, the smarter they’re supposed to become.

All that said, quantitative models don’t have as much of an advantage—if any—when making longer-term investment decisions.  If you’re going to hold a stock for a year, you only have 118 data points back to the year 1900.  As a result, some fundamental investors who hold stocks for 1 to 5 years (or longer) will likely continue to prosper, at least until artificial intelligence for long-term stock picking gets a lot better than it is now.  Most quant funds are focused on investments with short or very short holding periods.

***

As quantitative trading models become ever more widely adopted, the nature of the financial markets itself could change, introducing new risks.  Zuckerman:

Author and former risk manager Richard Bookstaber has argued that risks today are significant because the embrace of quant models is “system-wide across the investment world,” suggesting that future troubles for these investors would have more impact than in the past.  As more embrace quantitative trading, the very nature of financial markets could change.  New types of errors could be introduced, some of which have yet to be experienced, making them harder to anticipate.  Until now, markets have been driven by human behavior, reflecting the dominant roles played by traders and investors.  If machine learning and other computer models become the most influential factors in the markets, they may become less predictable and maybe even less stable, since human nature is roughly constant while the nature of this kind of computerized trading can change rapidly.

On the other hand, computerized trading could very well make markets more stable, argues Zuckerman:

The dangers of computerized trading are generally overstated, however.  There are so many varieties of quant investing that it is impossible to generalize about the subject.  Some quants employ momentum strategies, so they intensify the selling by other investors in a downturn.  But other approaches—including smart beta, factor investing, and style investing—are the largest and fastest-growing investment categories in the quant world.  Some of these practitioners have programmed their computers to buy when stocks get cheap, helping to stabilize the market.

It’s important to remember that market participants have always tended to pull back and do less trading during crises, suggesting that any reluctance by quants to trade isn’t so very different from past approaches.  If anything, markets have become more placid as quant investors have assumed dominant positions.  Humans are prone to fear, greed, and outright panic, all of which tend to sow volatility in financial markets.  Machines could make markets more stable, if they elbow out individuals governed by biases and emotions.  And computer-driven decision-making in other fields, such as the airline industry, has generally led to fewer mistakes.

***

Zuckerman notes Renaissance’s three-decade track record:

By the summer of 2019, Renaissance’s Medallion fund had racked up average annual gains, before investors fees, of about 66 percent since 1988, and a return after fees of approximately 39 percent.  Despite RIEF’s early stumbles, the firm’s three hedge funds open for outside investors have also outperformed rivals and market indexes.  In June 2019, Renaissance managed a combined $65 billion, making it one of the largest hedge-fund firms in the world, and sometimes represented as much as 5 percent of daily stock-trading volume, not including high-frequency traders.

Renaissance relies on the scientific method rather than human intuition and judgment:

…staffers embrace the scientific method to combat cognitive and emotional biases, suggesting there’s value to this philosophical approach when tackling challenging problems of all kinds.  They propose hypotheses and then test, measure, and adjust their theories, trying to let data, not intuition and instinct, guide them.

“The approach is scientific,” Simons says.  “We use very rigorous statistical approaches to determine what we think is underlying.”

Another lesson of the Renaissance experience is that there are more factors and variables influencing financial markets and individual investments than most realize or can deduce.  Investors tend to focus on the most basic forces, but there are dozens of factors, perhaps whole dimensions of them, that are missed.  Renaissance is aware of more of the forces that matter, along with the overlooked mathematical relationships that affect stock prices and other investments, than most anyone else.

However, Zuckerman observes:

For all the unique data, computer firepower, special talent, and trading and risk management expertise Renaissance has gathered, the firm only profits on barely more than 50 percent of its trades, a sign of how challenging it is to try to beat the market—and how foolish it is for most investors to try.

Simons and his colleagues generally avoid predicting pure stock moves.  It’s not clear any expert or system can reliably predict individual stocks, at least over the long term, or even the direction of financial markets.  What Renaissance does is try to anticipate stock moves relative to other stocks, to an index, to a factor model, and to an industry.

I disagree that no one can predict individual stocks.  Especially when their assets under management weren’t too large, Warren Buffett and his partner Charlie Munger clearly were more often right than wrong in predicting how individual businesses—and their associated stocks—were going to perform over time.  However, now that Berkshire Hathaway has hundreds of billions of dollars, including over one hundred billion in cash, Buffett and Munger are forced to look only at larger businesses, the stocks of which tend to be more efficiently priced.

In brief, if an investor focuses on microcap stocks (market caps of $300 million or less)—which are ignored by the vast majority of professional investors—it’s quite possible for someone who is patient, diligent, and reasonably intelligent to successfully pick stocks so as to generate performance in excess of the S&P 500 Index and the Russell Microcap Index.  Or an investor could apply a quantitative value approach to microcap stocks and also do better than the S&P 500 and the Russell Microcap Index.  That’s what the Boole Microcap Fund does.

Zuckerman concludes the chapter:

The Simons Foundation, with an annual budget of $450 million, had emerged as the nation’s second-largest private funder of research in basic science.  Math for America, the organization Simons helped found, provided annual stipends of $15,000 to over one thousand top math and science teachers in New York City.  It also hosted hundreds of annual seminars and workshops, creating a community of skilled and enthusiastic teachers.  There were signs the initiative was helping public schools retain the kinds of teachers who previously had bolted for private industry.

 

EPILOGUE

Zuckerman notes that by Spring 2019, Simons was focused on his two greatest challenges—curing autism, and discovering the origins of the universe and life itself:

True breakthroughs in autism research hadn’t been achieved and time was ticking by.  Six years earlier, the Simons Foundation had hired Louis Reichardt, a professor of physiology and neuroscience who was the first American to climb both Mount Everest and K2.  Simons handed Reichardt an even more daunting challenge: improve the lives of those with autism.

The foundation helped establish a repository of genetic samples from 2,800 families with at least one child on the autism spectrum, accelerating the development of animal models, a step towards potential human treatments.  By the spring of 2019, Simons’s researchers had succeeded in gaining a deeper understanding of how the autistic brain works and were closing in on drugs with the potential to help those battling the condition.

Zuckerman continues:

Simons was just as hopeful about making headway on a set of existential challenges that have confounded humankind from its earliest moments.  In 2014, Simons recruited Princeton University astrophysicist David Spergel, who is known for groundbreaking work measuring the age and composition of the universe.  Simons tasked Spergel with answering the eternal question of how the universe began.  Oh, a please try to do it in a few years, while I’m still around, Simons said.

Simons helped fund a $75 million effort to build an enormous observatory with an array of ultrapowerful telescopes in Chile’s Atacama Desert, a plateau 17,000 feet above sea level featuring especially clear, dry skies.  It’s an ideal spot to measure cosmic microwave radiation and get a good look into creation’s earliest moments.  The project, led by a group of eight scientists including Spergel and Brain Keating—an astrophysicist who directs the Simons Observatory and happens to be the son of Simons’s early partner, James Ax—is expected to be completed by 2022…

Many scientists assume the universe instantaneously expanded after creation, something they call cosmic inflation.  That event likely produced gravitational waves and twisted light, or what Keating calls “the fingerprint of the Big Bang.”…

…Subscribing to a view that time never had a starting point, Simons simultaneously supports work by Paul Steinhardt, the leading proponent of the noninflationary, bouncing model, an opposing theory to the Big Bang.

 

BOOLE MICROCAP FUND

An equal weighted group of micro caps generally far outperforms an equal weighted (or cap-weighted) group of larger stocks over time.  See the historical chart here:  https://boolefund.com/best-performers-microcap-stocks/

This outperformance increases significantly by focusing on cheap micro caps.  Performance can be further boosted by isolating cheap microcap companies that show improving fundamentals.  We rank microcap stocks based on these and similar criteria.

There are roughly 10-20 positions in the portfolio.  The size of each position is determined by its rank.  Typically the largest position is 15-20% (at cost), while the average position is 8-10% (at cost).  Positions are held for 3 to 5 years unless a stock approaches intrinsic value sooner or an error has been discovered.

The mission of the Boole Fund is to outperform the S&P 500 Index by at least 5% per year (net of fees) over 5-year periods.  We also aim to outpace the Russell Microcap Index by at least 2% per year (net).  The Boole Fund has low fees.

 

If you are interested in finding out more, please e-mail me or leave a comment.

My e-mail: jb@boolefund.com

 

 

 

Disclosures: Past performance is not a guarantee or a reliable indicator of future results. All investments contain risk and may lose value. This material is distributed for informational purposes only. Forecasts, estimates, and certain information contained herein should not be considered as investment advice or a recommendation of any particular security, strategy or investment product. Information contained herein has been obtained from sources believed to be reliable, but not guaranteed. No part of this article may be reproduced in any form, or referred to in any other publication, without express written permission of Boole Capital, LLC.

How Not to Be Wrong

September 11, 2022

Jordan Ellenberg has written a wonderful book, How Not to Be Wrong: The Power of Mathematical Thinking (Penguin, 2014).

Steven Pinker comments:

Like Lewis Carroll, George Gamow, and Martin Gardner before him, Jordan Ellenberg shows how mathematics can delight and stimulate the mind.  But he also shows that mathematical thinking should be in the toolkit of every thoughtful person—of everyone who wants to avoid fallacies, superstitions, and other ways of being wrong.

Here’s the outline:

    • When Am I Going to Use This?

PART I.  LINEARITY

    • One.  Less Like Sweden
    • Two.  Straight Locally, Curved Globally
    • Three.  Everyone is Obese
    • Four.  How Much Is That in Dead Americans?
    • Five.  More Pie Than Plate

PART II.  INFERENCE

    • Six.  The Baltimore Stockbroker and the Bible Code
    • Seven.  Dead Fish Don’t Read Minds
    • Eight.  Reductio Ad Unlikely
    • NineThe International Journal of Haruspicy
    • Ten.  Are You There God?  It’s Me, Bayesian Inference

PART III.  EXPECTATION

    • Eleven.  What to Expect When You’re Expecting to Win the Lottery
    • Twelve.  Miss More Planes!
    • Thirteen.  Where the Train Tracks Meet

PART IV.  REGRESSION

    • Fourteen.  The Triumph of Mediocrity
    • Fifteen.  Galton’s Ellipse
    • Sixteen.  Does Lung Cancer Make You Smoke Cigarettes?

PART V.  EXISTENCE

    • Seventeen.  There Is No Such Thing as Public Opinion
    • Eighteen.  “Out of Nothing I Have Created a Strange New Universe”
    • How to Be Right

Illustration by Sergii Pal

 

WHEN AM I GOING TO USE THIS?

Ellenburg tells the story of Abraham Wald.

This story, like many World War II stories, starts with the Nazis hounding a Jew out of Europe and ends with the Nazis regretting it…. He was the grandson of a rabbi and the son of a kosher baker, but the younger Wald was a mathematician almost from the start.  His talent for the subject was quickly recognized, and he was admitted to study mathematics at the University of Vienna, where he was drawn to subjects abstract and recondite even by the standards of pure mathematics: set theory and metric spaces.

In the mid-1930s, Austria was in economic distress and it wasn’t possible for a foreigner to be hired as a professor.  Wald ended up taking a job at the Cowles Commission, an economic institute then in Colorado Springs.  A few months later, Wald was offered a professorship of statistics at Columbia University.  When World War II came, the Statistical Research Group was formed.

The Statistical Research Group (SRG), where Wald spent much of World War II, was a classified program that yoked the assembled might of American statisticians to the war effort—something like the Manhattan Project, except the weapons being developed were equations, not explosives.

The mathematical talent at SRG was extraordinary.

Frederick Mosteller, who would later found Harvard’s statistics department, was there.  So was Leonard Jimmy Savage, the pioneer of decision theory and great advocate of the field that came to be called Bayesian statistics.  Norbert Wiener, the MIT mathematician and the creator of cybernetics, dropped by from time to time.  This was a group where Milton Friedman, the future Nobelist in economics, was often the fourth-smartest person in the room.

The smartest person in the room was usually Abraham Wald.

Planes need armor in order to lessen their chance of being shot down.  You don’t want too much armor due to the added weight, but you also don’t want too little armor, causing more planes to be shot down.  So there was a question about where to put extra armor on the planes.

The planes that came back had bullet holes per square foot distributed as follows:  1.11 for the engine, 1.73 for the fuselage, 1.55 for the fuel system, and 1.8 for the rest of the plane.

Given this knowledge, where should you put the extra armor?  At first glance, you might think the added armor should not be on the engine or fuel system, but on the fuselage and the rest of the plane.  However, Wald said the extra armor should go not where the bullet holes are, but where they aren’t:  on the engines.  Why?  Because you have to consider the planes that never made it back.  Those planes had bullet holes on the engines, which is why they never made it back.  Ellenberg comments:

If you go to the recover room at the hospital, you’ll see a lot more people with bullet holes in their legs than people with bullet holes in their chests.

That’s because the people shot in the chest generally don’t recover.  Ellenberg again:

One thing the American defense establishment has traditionally understood very well is that countries don’t win wars just by being braver… The winners are usually the guys who get 5% fewer of their planes shot down, or use 5% less fuel, or get 5% more nutrition into their infantry at 95% of the cost.  That’s not the stuff war movies are made of, but it’s the stuff wars are made of.  And there’s math every step of the way.

The reason Wald figured out where to put the armor was due to his mathematical training.

A mathematician is always asking, “What assumptions are you making?  Are they justified?”

The officers were assuming that the planes that returned were a random sample of all planes.  But this wasn’t the case.  There was no reason to assume the planes had an equal chance of survival no matter where they had been hit.

Ellenberg asserts:

Mathematics is the extension of common sense by other means.

Ellenberg again:

Mathematics is the study of things that come out a certain way because there is no other way they could possibly be.

For example:

…we have built-in mental systems for assessing the likelihood of an uncertain outcome.  But those systems are pretty weak and unreliable, especially when it comes to events of extreme rarity.  That’s when we shore up our intuition with a few sturdy, well-placed theorems and techniques, and make out of it a mathematical theory of probability.

(Gaussian normal distribution, Illustration by Peter Hermes Furian)

The specialized language in which mathematicians converse with one another is a magnificent tool for conveying complex ideas precisely and swiftly.  But its foreignness can create among outsiders the impression of a sphere of thought totally alien to ordinary thinking.  That’s exactly wrong.

Math is like an atomic-powered prosthesis that you attach to your common sense, vastly multiplying its reach and strength.  Despite the power of mathematics, and despite its sometimes forbidding notation and abstraction, the actual mental work involved is little different from the way we think about more down-to-earth problems.

Ellenberg continues:

Mathematics is not settled.  Even concerning the basic objects of study, like numbers and geometric figures, our ignorance is much greater than our knowledge.  And the things we do know were arrived at only after massive effort, contention, and confusion.  All this sweat and tumult is carefully screened off in your textbook.

Ellenberg then explains that mathematical facts can be simple or complicated, and they can be shallow or profound.  The focus of the book is on the simple yet profound.  These tools can help you not be wrong.

Ellenberg mentions his own experience:

As a graduate student, I dedicated myself to number theory, what Gauss called “the queen of mathematics,” the purest of the pure subjects, the sealed garden at the center of the convent, where we contemplated the same questions about numbers and equations that troubled the Greeks and have gotten hardly less vexing in the twenty-five hundred years since.

Ellenberg then adds:

But something funny happened.  The more abstract and distant from lived experience my research got, the more I started to notice how much math was going on in the world outside the walls.  Not Galois representations or cohomology, but ideas that were simpler, older, and just as deep… I started writing articles for magazines and newspapers about the way the world looked through a mathematical lens, and I found, to my surprise, that even people who said they hated math were willing to read them.

 

PART I: LINEARITY

ONE: LESS LIKE SWEDEN

Daniel J. Mitchell of the libertarian Cato Institute wrote a blog post in 2012, during the battle over the Affordable Care Act.  The blog post was titled, “Why Is Obama Trying to Make America More Like Sweden when Swedes Are Trying to Be Less Like Sweden?”  Mitchell writes: “If Swedes have learned from their mistakes and are now trying to reduce the size and scope of government, why are American politicians determined to repeat those mistakes?”

The answer has to do with a nonlinear relationship between prosperity and Swedishness, writes Ellenberg.  It’s false to say that reducing the size of government automatically increases prosperity, just as it’s false to say that increasing the size of government automatically increases prosperity.  Arguably, there’s an optimal size for government that America is below and that Sweden is above.

Illustration by Oleksandra Drypsiak

 

TWO: STRAIGHT LOCALLY, CURVED GLOBALLY

Ellenberg continues the theme of nonlinearity:

You might not have thought you needed a professional mathematician to tell you that not all curves are straight lines.  But linear reasoning is everywhere.  You’re doing it every time you say that if something is good to have, having more of it is even better.

Ellenberg points out that if you look at one section of a curve very closely, it looks like a line.  Ellenberg explains calculus by using the example of a missile that follows a curved path due to gravity:

Now here’s the conceptual leap.  Newton said, look, let’s go all the way.  Reduce your field of view until it’s infinitesimal—so small that it’s smaller than any size you can name, but not zero.  You’re studying the missile’s arc, not over a very short time interval, but at a single moment.  What was almost a line becomes exactly a line.  And the slope of this line is what Newton called the fluxion, and what we’d now call the derivative.

(Projectile motion illustration by Ayush12gupta, via Wikimedia Commons)

Ellenberg introduces Zeno’s paradox.  In order to walk from one place to another place, first you must walk half the distance.  Then you must walk half of the remaining distance.  After that, in order to get to your destination, you must walk half of the remaining distance.  Thus it seems you can never reach your destination.  To go anywhere, first you must go half way there, then you must cover half the remaining distance, ad infinitum.  Ellenberg observes that all motion is ruled out.  To wave your hand, first you must raise your hand.  To do that, first you must raise it halfway, etc.

What’s the solution?  Ellenberg says to consider the infinite series:

1/2 + 1/4 + 1/8 + 1/16 + 1/32 + ….

If you add up the first twenty terms, you get 0.999999.  If you keep adding terms, you get 0.9999….., which doesn’t end.  But is 0.9999… equal to 1?

Ellenberg says to consider that

0.3333…. = 1/3

Multiply both sides by 3:

0.9999…. = 1

You could also do it this way, multiplying 0.9999…. by 10:

10 x (0.9999…) = 9.9999….

Subtract the decimal from both sides:

10 x (0.9999…) – 1 x (0.9999…) = 9.9999… – 0.9999…

9 x (0.9999…) = 9

This still doesn’t fully answer the question because we assume that 0.3333… = 1/3, which one could argue is not quite true.

Ellenberg presents another brain teaser:

1 + 2 + 4 + 8 + 16 + 32 + ….

What does this series equal?  It must be infinite, right?

What if you multiplied it by 2:

2 x (1 + 2 + 4 + 8 + …) = 2 + 4 + 8 + 16 + …

It looks like the same series but without the 1 at the beginning.  That implies:

2 x (1 + 2 + 4 + 8 + …) – 1 x (1 + 2 + 4 + 8 + …) = –1

Which means:

1 + 2 + 4 + 8 + … = –1

Really?  Here’s another interesting example:

1 – 1 + 1 – 1 + 1 – 1 + …

It looks like:

(1 – 1) + (1 – 1) + (1 – 1) + … = 0 + 0 + 0 + … = 0

But it also can be written:

1 – (1 – 1) – (1 – 1) – (1 – 1) = 1 – 0 – 0 – 0 = 1

If you define T as:

T = 1 – 1 + 1 – 1 + 1 – 1 + …

Then take the negative of both sides:

–T = – 1 + 1 – 1 + 1 … = T – 1

Then you get –2T = –1, or T = 1/2.

Now back to the infinite series:

0.9999….

Augustin-Louis Cauchy introduced the idea of limit into calculus in the 1820s.  Ellenberg:

The sum 0.9 + 0.09 + 0.009 + … gets closer and closer to 1 the more terms you add.  And it never gets any farther away.  No matter how tight a cordon we draw around the number 1, the sum will eventually, after some finite number of steps, penetrate it, and never leave.  Under those circumstances, Cauchy said, we should simply define the value of the infinite sum to be 1.  And then he worked very hard to prove that committing oneself to this definition didn’t cause horrible contradictions to pop up elsewhere.  By the time this labor was done, he’s constructed a framework that made Newton’s calculus completely rigorous.  When we say a curve looks locally like a straight line at a certain angle, we now mean more or less this: as you zoom in tighter and tighter, the curve resembles the given line more and more closely.

So we can take 1 and 0.999… to be equal.  Ellenberg concludes:

One of the great joys of mathematics is the incontrovertible feeling that you’ve understood something the right way, all the way down to the bottom; it’s a feeling I haven’t experienced in any other sphere of mental life.

 

THREE: EVERYONE IS OBESE

Ellenberg read an article about Americans getting more obese.  The article concludes that if this trend continues, ALL Americans will be obese by 2048.

As another example of linear regression, Ellenberg asks the reader to consider the relationship between SAT scores and tuition.  Generally, the higher the average SAT score, the higher the tuition.  What is linear regression?

…if you replace the actual tuition with the estimate the line suggests, and then you compute the difference between the estimated and the actual tuition for each school, and then you square each of these numbers, and you add all those squares up, you get some kind of total measure of the extent to which the line misses the points, and you choose the line that makes this measure as small as possible.  This business of summing squares smells like Pythagoras, and indeed the underlying geometry of linear regression is no more than Pythagoras’s theorem transposed and upgraded to a much-higher-dimensional setting…

Here’s an example of linear regression:

(Least squares fit by Prof. Boykin, via Wikimedia Commons)

Ellenberg continues:

Linear regression is a marvelous tool, versatile, scalable, and as easy to execute as clicking a button on your spreadsheet.  You can use it for data involving two variables… but it works just a well for three variables, or a thousand.  Whenever you want to understand which variables drive which other variables, and in which direction, it’s the first thing you reach for.  And it works on any data set at all.

However, linear regression is a tool that can be misused if it is applied to phenomena that aren’t linear.

Back to obesity: Overweight means having a body-mass index of 25 or higher.

In the early 1970s, just under half of Americans had a BMI that high.  By the early 1990s that figure had risen to almost 60%, and by 2008 almost three-quarters of the U.S. population was overweight.

If you extrapolate the trend, you conclude that ALL Americans will be overweight by 2048.  That doesn’t make sense.

 

FOUR: HOW MUCH IS THAT IN DEAD AMERICANS?

Ellenberg writes:

How bad is the conflict in the Middle East?  Counterterrorism specialist Daniel Byman of Georgetown University lays down some cold, hard numbers in Foreign Affairs: “The Israeli military reports that from the start of the second intifada [in 2000] through the end of October 2005, Palestinians killed 1,074 Israelis and wounded 7,520—astounding figures for such a small country, the proportional equivalent of more than 50,000 dead and 300,000 wounded for the United States.”

Ellenberg adds:

Eventually (or perhaps immediately?) this reasoning starts to break down.  When there are two men left in the bar at closing time, and one of them coldclocks the other, it is not the equivalent in context to 150 million Americans getting simultaneously punched in the face.

Or: when 11% of the population of Rwanda was wiped out in 1994, all agree that it was among the worst crimes of the century.  But we don’t describe the bloodshed there by saying, “In the context of 1940s Europe, it was nine times as bad as the Holocaust.”

As a proportion of the population, South Dakota has a high degree of brain cancer while North Dakota has a low degree of it.  Similarly, Maine has a high degree of brain cancer while Vermont has a low degree of it.  This seems strange.

To understand what’s doing on, Ellenberg considers coin flipping.  If you flip one coin ten times, you may get 8 or 9 heads, which is a proportion of 80% or 90%.  But if you flip a coin one hundred times, you’ll never get 80% heads unless you repeat the flipping a few billion times.

Photo by Ronstik

What causes a large number of coin tosses to move toward 50%?  The Law of Large Numbers.  Ellenberg:

…if you flip enough coins, there’s only the barest chance of getting as many as 51%.  Observing a highly unbalanced result in ten flips is unremarkable; getting the same proportional imbalance in a hundred flips would be so startling as to make you wonder whether someone has mucked with your coins.

In 1756, Abraham de Moivre published The Doctrine of Chances.  De Moivre wanted to know how close to 50% heads you would get if you flipped a large number of coins.

De Moivre’s insight is that the size of the typical discrepancy is governed by the square root of the number of coins you toss.  Toss a hundred times as many coins as before and the typical discrepancy grows by a factor of 10—at least, in absolute terms.  As a proportion of the total number of tosses, the discrepancy shrinks as the number of coins grows, because the square root of the number of coins grows much more slowly than does the number of coins itself.

…if you want to know how impressed to be by a good run of heads, you can ask how many square roots away from 50% it is.  The square root of 100 is 10.  So when I got 60 heads in 100 tries, that was exactly one square root away from 50-50.  The square root of 1,000 is about 31; so when I got 538 heads in 1,000 tries, I did something even more surprising, even though I got only 53.8% heads in the latter case and 60% heads in the former.

But de Moivre wasn’t done.  He found that the discrepancies from 50-50, in the long run, always tend to form themselves into a perfect bell curve, or, as we call it in the biz, the normal distribution.

Ellenberg continues:

The bell curve… is tall in the middle and very flat near the edges, which is to say that the farther a discrepancy is from zero, the less likely it is to be encountered.  And this can be precisely quantified.

One common misperception is the so-called law of averages (which Ellenberg points out is not well-named because laws should be true but the law of averages is false).  If you flip ten heads in a row, what are the odds that the next flip will be heads?  Based on the Law of Large Numbers, you might think the next flip is extremely likely to be tails.  But it’s not.  It’s still only 50% likely to be tails and 50% likely to be heads.  Ellenberg explains:

The way the overall proportion settles down to 50% isn’t that fate favors tails to compensate for the heads that have already landed; it’s that those first ten flips become less and less important the more flips we make.  If I flip the coin a thousand more times, and get about half heads, then the proportion of heads in the first 1,010 flips is also going to be close to 50%.

So can wars and other violent events be compared based on proportion of population?  Ellenberg concludes the chapter:

Most mathematicians would say that, in the end, the disasters and the atrocities of history form what we call a partially ordered set.  That’s a fancy way of saying that some pairs of disasters can be meaningfully compared and others cannot.  This isn’t because we don’t have accurate enough death counts, or firm enough opinions as to the relative merits of being annihilated by a bomb versus dying of war-induced famine.  It’s because the question of whether one war was worse than another is fundamentally unlike the question of whether one number is bigger than another.  The latter question always has an answer.  The former does not.

 

FIVE: MORE PIE THAN PLATE

Ellenberg writes:

Between 1990 and 2008, the U.S. economy gained a net 27.3 million jobs.  Of those, 26.7 million, or 98%, came from the “nontradable sector”: the part of the economy including things like government, health care, retail, and food service, which can’t be outsourced and which don’t produce goods to be shipped overseas.

…So [is] growth as concentrated in the nontradable part of the economy as it could possibly be?  That’s what it sounds like—but that’s not quite right.  Jobs in the tradable sector grew by a mere 620,000 between 1990 and 2008, that’s true.  But it could have been worse—they could have declined!  That’s what happened between 2000 and 2008; the tradable sector lost about 3 million jobs, while the nontradable sector added 7 million.  So the nontradable sector accounted for 7 million jobs out of the total gain of 4 million, or 175%!

The slogan to live by here is:

Don’t talk about percentages of numbers when the numbers might be negative.

Ellenberg gives the example of a coffee shop.  Say he lost $500 on coffee, but made $750 on a pastry case and $750 on a CD Rack.  Overall, he made $1,000.  75% of profits came from the pasty case.  Or you could say that 75% of profits came from the CD rack.  To say either one is misleading.

This problem doesn’t happen with numbers that have to be positive, like revenues.

Consider growth in income from 2009 to 2010, says Ellenberg.  93% of additional income went to the top 1% of taxpayers.  17% of additional income went to those in the top 10%, but not the top 1%, of taxpayers.  How does that make sense?  Again, because of a negative number: the bottom 90% of taxpayers saw their incomes move lower.

 

PART II: INFERENCE

SIX: THE BALTIMORE STOCKBROKER AND THE BIBLE CODE

Ellenberg observes that mathematics is used for a wide range of things, from when the next bus can be expected to what the universe looked like three trillionths of a second after the Big Bang.  But what about questions concerning God and religion?  Ellenberg:

Never underestimate the territorial ambitions of mathematics!  You want to know about God?  There are mathematicians on the case.

The rabbinical scholar focuses on the Torah, notes Ellenberg.  A group of researchers at Hebrew University—senior professor of mathematics Eliyahu Rips, graduate student in computer science Yoav Rosenberg, and physicist Doron Witztum—started examining the Torah.  Specifically, they looked at “equidistant letter sequence,” or ELS.  The question they asked was:

Do the names of the rabbis appear in equidistant letter sequences unusually close to their birth and death dates?

Put differently: Did the Torah know the future?

Ellenberg:

First they searched the book of Genesis for ELSs spelling out the rabbis’ names and dates, and computed how close in the text the sequences yielding the names were to the ones yielding the corresponding dates.  Then they shuffled the thirty-two dates, so that each one was now matched with a random rabbi, and they ran the test again.  Then they did the same thing a million times.  If there were no relation in the Torah’s text between the names of the rabbis and the corresponding dates, you’d expect the true matching between rabbis and dates to do about as well as one of the random shuffles.  That’s not what they found.  The correct association ended up very near the top of the rankings, notching the 453rd highest score among the 1 million contenders.

American journalist Michael Drosnin heard about the Witztum paper.  Drosnin started looking for ELSs, but without scientific constraint.  Drosnin published The Bible Code, which claimed to predict Yitzhak Rabin’s assassination, the Gulf War, and the 1994 collision of Comet Shoemaker-Levy 9 with Jupiter.  Rips, Rosenberg, and Witztum denounced Drosnin’s ad hoc method.  But The Bible Code became a bestseller.

At the same time as the bible codes were being accepted by the public, the Witztum paper came under severe criticism from mathematicians including Shlomo Sternberg of Harvard University.  To understand the criticism, Ellenberg tells the story of the Baltimore stockbroker.

What if you got an unsolicited letter from a Baltimore stockbroker saying that a certain stock would rise, and then it does?  Further assume that each week, for ten weeks, you get a new letter predicting some stock to rise or fall, and each week, the prediction is correct.  The eleventh week, the Baltimore stockbroker asks you to invest money with him.

Now, what are the odds that the Baltimore stockbroker got ten straight predictions right due to chance alone?  Those odds can be computed:

(1/2) x (1/2) x (1/2) x (1/2) x (1/2) x (1/2) x (1/2) x (1/2) x (1/2) x (1/2) = 1/1024

However, what you didn’t know was that the Baltimore stockbroker mailed 10,240 letters the first week, half predicting a rise in a specific stock and half predicting a fall in that same stock.  So that first week, there were 5,120 correct predictions.  At that point, the stockbroker mails out 5,120 new letters, half of which predict a rise in some stock and half of which predict a fall.  After that second week, there are 2,560 people who have seen two correct predictions in a row.  And thus the Baltimore stockbroker continues until, after ten weeks, there are 10 people who received by mail 10 straight correction predictions.

Some companies will launch multiple mutual funds at the same time, experimenting with different strategies before advertising any fund to the public.  The funds that do well during this initial period—called incubation—are then marketed to the public, while the funds that don’t do well are quietly closed (and often these funds are not noticed by the public).  Subsequently, the surviving funds do not perform better than the median, which suggests the part of their early outperformance was simply due to chance.

(Photo by Martinlisner)

The general lesson, writes Ellenberg, is:

Improbable things happen a lot.

In other words, a great many things are due largely to chance, therefore there will always be improbable things happening.

Criticism of the Witztum paper came from Brendan McKay, an Australian computer scientist, and Dror Bar-Natan, an Israeli mathematician then at Hebrew University.  They pointed out that the rabbis didn’t have birth certificates or passports, so they were known by many different names.  Why did the Witztum paper, for each rabbi studied, use specific appellations but not others?  Ellenberg:

McKay and Bar-Natan found that wiggle room in the choices of names led to drastic changes in the quality of the results.  They made a different set of choices about the appellations of the rabbis; their choices, according to biblical scholars, make just as much sense as the ones picked by Witztum… And they found that with the new list of names, something quite amazing transpired.  The Torah no longer seemed to detect the birth and death dates of the rabbinic notables.  But the Hebrew edition of War and Peace nailed it, identifying the rabbis with their correct dates about as well as the book of Genesis did in the Witztum paper.

What’s going on?  Ellenberg:

It is very unlikely that any given set of rabbinic appellations is well matched to birth and death dates in the book of Genesis.  But with so many ways of choosing the names, it’s not at all improbable that among all the choices there would be one that made the Torah look uncannily prescient.

Obviously it’s also not the case that Tolstoy composed his novel with the names of rabbis concealed in it, to be later revealed when modern Hebrew was invented and the novel was translated into it.

 

SEVEN: DEAD FISH DON’T READ MINDS

At the 2009 Organization for Human Brain Mapping conference in San Francisco, UC Santa Barbara neuroscientist Craig Bennett presented his finding that a dead salmon was able to read human emotions.

A dead fish, scanned in an fMRI device, was shown a series of photographs of human beings, and was found to have a surprisingly strong ability to correctly assess the emotions the people in the pictures displayed.

What?!  Here’s what’s going on:

…the nervous system is a big place, with tens of thousands of voxels to choose from.  The odds that one of those voxels provides data matching up well with the photos is pretty good… The point of Bennett’s paper is to warn that the standard methods of assessing results, the way we draw our thresholds between a real phenomenon and random static, come under dangerous pressure in this era of massive data sets, effortlessly obtained.  We need to think very carefully about whether our standards for evidence are strict enough, if the empathetic salmon makes the cut.

Ellenberg adds:

The really surprising result of Bennett’s paper isn’t that one or two voxels in a dead fish passed a statistical test; it’s that a substantial proportion of the neuroimaging articles he surveyed didn’t use statistical safeguards (known as “multiple comparisons correction”) that take into account the ubiquity of the improbable.  Without those corrections, scientists are at serious risk of running the Baltimore stockbroker con, not only on their colleagues but on themselves.

The null hypothesis is the hypothesis that the intervention you’re studying has no effect, notes Ellenberg.

Illustration by Hafakot

To test your intervention, you have to run a null hypothesis significance test.  Ellenberg:

It goes like this.  First, you have to run an experiment.  You might start with a hundred subjects, then randomly select half to receive your proposed wonder drug while the other half gets a placebo…

From here, the protocol might seem simple: if you observe fewer deaths among the drug patients than the placebo patients, declare victory and file a marketing application with the FDA.  But that’s wrong.  It’s not enough that the data be consistent with your theory; they have to be inconsistent with the negation of your theory, the dreaded null hypothesis.

Assuming the null hypothesis, the chance of death (assume 10%) is exactly the same for the patients who got the drug and the patients who got the placebo.  That doesn’t mean that exactly five patients will die in each category.  Under the null hypothesis, Ellenberg says there is:

    • 13.3% chance equally many drug and placebo patients die
    • 43.3% chance fewer placebo patients than drug patients die
    • 43.3% chance fewer drug patients than placebo patients die

So drug patients doing better than placebo patients is not necessarily significant.  But what if drug patients do A LOT better, asks Ellenberg.  Assume, for example, that none of the drug patients die.  What are the odds that the could happen under the null hypothesis?

Under the null hypothesis, here’s a 90% chance that a patient will survive.  What are the odds that all fifty survive?

0.9 x 0.9 x 0.9 x … fifty times! … x 0.9 x 0.9 = 0.00515…

Under the null hypothesis, there is only one chance in two hundred of getting results this good, observes Ellenberg.

So here’s the procedure for ruling out the null hypothesis…:

    • Run an experiment.
    • Suppose the null hypothesis is true, and let p be the probability (under that hypothesis) of getting results as extreme as those observed.
    • The number p is called the p-value. If it is very small, rejoice; you get to say your results are statistically significant.  If it is large, concede that the null hypothesis has not been ruled out.

 

EIGHT: REDUCTIO AD UNLIKELY

Ellenberg points out that assuming the null hypothesis, which we believe is false, might seem questionable.  But the reductio ad absurdum goes all the way back to Aristotle.  If a hypothesis implies a falsehood, then the hypothesis must be false.  The reduction ad absurdum looks like this:

    • Suppose the hypothesis H is true.
    • It follows from H that a certain fact F cannot be the case.
    • But F is the case.
    • Therefore, H is false.

Ellenberg then describes what he calls a reductio ad unlikely:

    • Suppose the null hypothesis H is true.
    • It follows from H that a certain outcome O is very improbable (say, less than Fisher’s 0.05 threshold).
    • But O was actually observed.
    • Therefore, H is very improbable.

Illustration by Ctitze

 

NINE: THE INTERNATIONAL JOURNAL OF HARUSPICY

Ellenberg writes about haruspicy:

…If the null hypothesis is always true—that is, if haruspicy is undiluted hocus-pocus—then only 1 in 20 experiments will be publishable.

And yet there are hundreds of haruspices, and thousands of ripped-open sheep, and even one in twenty divinations provides plenty of material to fill each issue of the journal with novel results, demonstrating the efficacy of the methods and the wisdom of the gods.  A protocol that worked in one case and gets published usually fails when another haruspex tries it; but experiments without statistically significant results don’t get published, so no one ever finds out about the failure to replicate.  And even if word starts getting around, there are always small differences the experts can point to that explain why the follow-up study didn’t succeed; after all, we know the protocol works, because we tested it and it had a statistically significant effect!

Ellenberg then makes his main point:

Modern medicine and social science are not haruspicy.  But a steadily louder drum circle of dissident scientists has been pounding out an uncomfortable message in recent years: there’s probably a lot more entrail reading in the sciences than we’d like to admit.

The loudest drummer is John Ioannidis, a Greek high school math star turned biomedical researcher whose 2005 paper “Why Most Published Research Findings Are False” touched off a fierce bout of self-criticism (and a second wave of self-defense) in the clinical sciences… Ioannidis takes seriously the idea that entire specialties of medical research are “null fields,” like haruspicy, in which there are simply no actual effects to be found.  “It can be proven,” he writes, “that most claimed research findings are false.”

Photo by Ekaterina79

Ellenberg:

…In a 2012 study, scientists at the California biotech company Amgen set out to replicate some of the most famous experimental results in the biology of cancer, fifty-three studies in all.  In their independent trials, they were able to reproduce only six.

How can this have happened?  It’s not because genomicists and cancer researchers are dopes.  In part, the replicability crisis is simply a reflection of the fact that science is hard and that most ideas we have are wrong—even most of those ideas that survive a first round of prodding.

Ellenberg again:

Suppose you tested twenty genetic markers to see whether they were associated with some disorder of interest, and you found just one result that achieved p < .05 significance.  Being a mathematical sophisticate, you’d recognize that one success in twenty is exactly what you’d expect if none of the markers had any effect…

All the more so if you tested the same gene, or the green jelly bean, twenty times and got a statistically significant effect just once.

But what if the green jelly bean were tested twenty times by twenty different research groups in twenty different labs?  Nineteen of the labs find no significant statistical effect.  They don’t write up their results… The scientists in the twentieth lab, the lucky ones, find a statistically significant effect, because they got lucky—but they don’t know they got lucky.

It can be difficult for scientists when the results seem to be statistically insignificant.  Ellenberg:

If you run your analysis and get a p-value of .06, you’re supposed to conclude that your results are statistically insignificant.  But it takes a lot of mental strength to stuff years of work in the file drawer… Give yourself license to tweak and shade the statistical tests you carry out on your results, and you can often get that .06 down to a .04.  Uri Simonsohn, a professor at Penn who’s a leader in the study of replicability, calls these practices “p-hacking.”  Hacking the p isn’t usually as crude as I’ve made it out to be, and it’s seldom malicious.  The p-hackers truly believe in their hypotheses, just as the Bible coders do, and when you’re a believer, it’s easy to come up with reasons that the analysis that gives a publishable p-value is the one you should have done in the first place.

But everybody knows it’s not really right.

Replication is central to science.  Ellenberg comments:

But even studies that could be replicated often aren’t.  Every journal wants to publish a breakthrough finding, but who wants to publish the paper that does the same experiment a year later and gets the same result?  Even worse, what happens to papers that carry out the same experiment and don’t find a significant result?  For the system to work, those experiments need to be made public.  Too often they end up in the file drawer instead.

But the culture is changing.  Reformers with loud voices like Ioannidis and Simonsohn, who speak both to the scientific community and to the broader public, have generated a new sense of urgency about the danger of descent into large-scale haruspicy.  In 2013, the Association for Psychological Science announced that they would start publishing a new genre of article, called Registered Replication Reports.  These reports, aimed at reproducing the effects reported in widely cited studies, are treated differently from usual papers in a crucial way: the proposed experiment is accepted for publication before the study is carried out.

 

TEN: ARE YOU THERE GOD?  IT’S ME, BAYESIAN INFERENCE

While we can predict the course of an asteroid better as we get ever more data, there may be hard limits on how far into the future meteorologists can predict the weather.  The weather is chaotic, and a sea gull flapping its wings could alter the weather forever.  Ellenberg:

Is human behavior more like an asteroid or more like the weather?  It surely depends on what aspect of human behavior you’re talking about.  In at least one respect, human behavior ought to be even harder to predict than the weather.  We have a very good mathematical model for weather, which allows us at least to get better at short-range predictions when given access to more data, even if the inherent chaos of the system inevitably wins out.  For human action we have no such model and may never have one.  That makes the prediction problem massively harder.

Ellenberg asks whether Facebook can predict that someone is a terrorist.

on Facebook list Not on list
terrorist 10 9,990
not terrorist 99,990 199,890,010

What if your neighbor ends up on the Facebook list?  Ellenberg:

The null hypothesis is that your neighbor is not a terrorist.  Under that hypothesis—that is, presuming his innocence—the chance of him showing up on the Facebook red list is a mere 0.05%, well below the 1-in-20 threshold of statistical significance.  In other words, under the rules that govern the majority of contemporary science, you’d be justified in rejecting the null hypothesis and declaring your neighbor a terrorist.

Except there’s a 99.99% chance he’s not a terrorist.

Ellenberg continues:

On the one hand, there’s hardly any chance that an innocent person will be flagged by the algorithm.  At the same time, the people the algorithm points to are almost all innocent.  It seems like a paradox, but it’s not.  It’s just how things are…

Here’s the crux.  There are really two questions you can ask.  They sound kind of the same, but they’re not.

Question 1: What’s the chance that a person gets put on Facebook’s list, given that they’re not a terrorist?

Question 2: What’s the chance that a person’s not a terrorist, given that they’re on Facebook’s list?

One way you can tell these two questions are different is that they have different answers.  Really different answers.  We’ve already seen that the answer to the first question is about 1 in 2,000, while the answer to the second is 99.99%.  And it’s the answer to the second question that you really want.

Ellenberg:

The p-value is the answer to the question

“The chance that the observed experimental result would occur, given that the null hypothesis is correct.”

But what we want to know is the other conditional probability:

“The chance that the null hypothesis is correct, given that we observed a certain experimental result.”

Ellenberg adds:

The danger arises precisely when we confuse the second quantity for the first.  And this confusion is everywhere, not just in scientific studies.  When the district attorney leans into the jury box and announces, “There is only a one in five million, I repeat, a ONE IN FIVE MILLLLLLLION CHANCE that an INNOCENT MAN would match the DNA sample found at the scene,” he is answering question 1, How likely would an innocent person be to look guilty?  But the jury’s job is to answer question 2, How likely is this guilty-looking defendant to be innocent?

In Bayesian inference, you start out with your best guess—the a priori probability that something is true—and then, once you get further evidence, you end up with a posterior probability.  Consider the example of the neighbor being on Facebook’s terrorist list.

The neighbor’s presence on the list really does offer some evidence that he’s a potential terrorist.  But your prior for that hypothesis ought to be very small, because most people aren’t terrorists.  So, despite the evidence, your posterior probability remains small as well, and you don’t—or at least shouldn’t—worry.

Ellenberg writes:

For those who are willing to adopt the view of probability as degree of belief, Bayes’s theorem can be seen not as a mere mathematical equation but as a form of numerically flavored advice.  It gives us a rule, which we may choose to follow or not, for how we should update our beliefs about things in the light of new observations.  In this new, more general form, it is naturally the subject of much fiercer disputation.  There are hard-core Bayesians who think that all our beliefs should be formed by strict Bayesian computations, or at least as strict as our limited cognition can make them; others think of Bayes’s rule as more of a loose qualitative guideline.

Here is Bayes’s rule:

(Photo by mattbuck, via Wikimedia Commons)

Ellenberg quotes Sherlock Holmes:

“It is an old maxim of mine that when you have excluded the impossible, whatever remains, however improbable, must be the truth.”

Ellenberg says Holmes should have said the following:

“It is an old maxim of mine that when you have excluded the impossible, whatever remains, however improbable, must be the truth, unless the truth is a hypothesis it didn’t occur to you to consider.”

Ellenberg gives the example of GOD vs. NO GOD.  There are other possibilities, like GODS (plural), which could help explain creation.  Ellenberg:

…Another theory with some adherents is SIMS, where we’re not actually people at all, but simulations running on an ultracomputer built by other people.  That sounds bizarre, but plenty of people take the idea seriously (most famously, the Oxford philosopher Nick Bostrom), and on Bayesian grounds, it’s hard to see why you shouldn’t.  People like to build simulations of real-world events; surely, if the human race doesn’t extinguish itself, our power to simulate will only increase, and it doesn’t seem crazy to imagine that those simulations might one day include conscious entities that believe themselves to be people.

All that said, Ellenberg notes that it’s probably best to arrive at faith—or to discard it—in a non-quantitative way.  And people should stick to, “I do believe in God,” or “I don’t believe in God,” or “I’m not sure.”

 

PART III: EXPECTATION

ELEVEN: WHAT TO EXPECT WHEN YOU’RE EXPECTING TO WIN THE LOTTERY

Ellenberg notes:

During the Revolutionary War, both the Continental Congress and the governments of the states established lotteries to fund the fight against the British.  Harvard, back in the days before it enjoyed a nine-figure endowment, ran lotteries in 1794 and 1810 to fund two new college buildings.

Like “statistical significance,” the term “expected value” is misleading.  If you bet $10 on a dog that has a 10% chance of winning, then the expected value is:

(10% x $100) + (90% x $0) = $10

The expected value for the $10 bet is $10.  But that’s not what you expect.  You expect either $0 or $100.  Ellenberg states that “average value” might be a better term than “expected value.”  If you make one thousand $10 bets—with each bet having a 10% chance of winning $100 and a 90% chance of winning $0—then you would expect to make about $10,000, which equals the total amount that you bet.  Over time, if you keep repeating this bet, you expect to come out even.

What is your expected value from playing Powerball?  Assuming the jackpot is $100 million, your expected value is:

100 million / 175 million + 1 million / 5 million + 10,000 / 650,000 + 100 / 19,000 + 100 / 12,000 + 7 / 700 + 7 / 360 + 4 / 110 + 4 / 55

All the different terms represent different amounts that you can win playing Powerball with the jackpot at $100 million.  (These smaller prizes keep people feeling that the game is worth playing.)  If you work out that math, your expected value from playing is just under 94 cents per $2 ticket.  For each $2 ticket you buy, on average you expect to get back just under 94 cents.  It would seem that the game is not worth playing, but what if the jackpot, call it J, is higher?  Your expected value is:

J / 175 million + 1 million / 5 million + 10,000 / 650,000 + 100 / 19,000 + 100 / 12,000 + 7 / 700 + 7 / 360 + 4 / 110 + 4 / 55

This simplifies to:

J / 175 million + 36.7 cents

The breakeven threshold is a bit over J = $285 million.  So if the jackpot is greater than $285 million, then you expect to win more than $2 for each $2 bet.  It makes sense not only to play, but to buy as many $2 tickets as you can reasonably afford.

But this assumes that all the other players fail to win the jackpot.  Assume there are 75 million players total.  Assume you win and that the jackpot is $337 million.  What are the odds that everyone else loses?  It is (174,999,999 / 175,000,000) multiplied by itself 75 million times, which is 0.651… (65.1%).  This means there’s about a 35% chance that someone else will win, which means you would have to share the jackpot.  Your expected payoff is:

65% x $337 million + 35% x $168 million = $278 million

$278 million is below the $285 million threshold, so the possibility of sharing the jackpot means that the game is no longer worth playing (assuming a $337 million jackpot and 75 million players).

Photo by Gajus

Are lotteries always bad bets?  No.  Consider the story of Cash WinFall in Massachusetts.  Here’s the prize distribution on a normal day:

match all 6 numbers 1 in 9.3 million variable jackpot
match 5 of 6 1 in 39,000 $4,000
match 4 of 6 1 in 800 $150
match 3 of 6 1 in 47 $5
match 2 of 6 1 in 6.8 free lottery ticket

Assume the jackpot is $1 million.  Then the expected return on a $2 ticket is:

($1 million / 9.3 million) + ($4,000 / 39,000) + ($150 / 800) + ($5 / 47) + ($2 / 6.8) = 79.8 cents

Each $2 bet would return about 80 cents, which is not a good bet.  However, roll-down days are different.  (Roll-downs happen when nobody wins the jackpot, so the prize money is rolled down.)  On February 7, 2005, nobody won the $3 million dollar jackpot.  So the money was rolled down:

The state’s formula rolled $600,000 to the match-5 and match-3 prize pools and $1.4 million into the match-4s.  The probability of getting 4 out of 6 WinFall numbers right is about 1 in 800, so there must have been about 600 match-4 winners that day out of the 470,000 players.  That’s a lot of winners, but $1.4 million dollars is a lot of money… In fact, you’d expect the payout for matching 4 out of 6 numbers that day to be around $2,385.  That’s a much more attractive proposition than the measly $150 you’d win on a normal day.  A 1-in-800 chance of a $2,385 payoff has an expected value of

$2385 / 800 = $2.98

The match-4 prize alone makes the game profitable.  If you add in the other payoffs, the ticket is worth

$50,000 / 39,000 + $2385 / 800 + $60 / 47 = $5.53

So for each $2 you invest, you expect to get back $5.53 on average.  To be clear: If you only bought one ticket, you probably wouldn’t win even though the average payoff is positive.  However, if you bought one thousand tickets, or ten thousand, then you would almost certainly earn a profitable return of about $5.53 per $2 ticket purchased.

February 7, 2005, is when James Harvey, an MIT senior doing an independent study on the merits of various state lottery games, realized that Massachusetts had accidentally created a highly profitable investment opportunity.  Harvey got a group of MIT friends together, and they purchased a thousand tickets.  Overall, they tripled their investment.  Ellenberg:

It won’t surprise you to hear that Harvey and his co-investors didn’t stop playing Cash WinFall.  Or that he never did get around to finishing that independent study—at least not for course credit.  In fact, his research project quickly developed into a thriving business.  By summer, Harvey’s confederates were buying tens of thousands of tickets at a time… They called their team Random Strategies, though their approach was anything but scattershot; the name referred to Random Hall, the MIT dorm where Harvey had originally cooked up his plan to make money on WinFall.

And the MIT students weren’t alone.  At least two more betting clubs formed up to take advantage of the WinFall windfall.  Ying Zhang, a medical researcher in Boston with a PhD from Northeastern, formed the Doctor Zhang Lottery Club… Before long, the group was buying $300,000 worth of tickets for each roll-down.  In 2006, Doctor Zhang quit doctoring to devote himself full-time to Cash WinFall.

Still another betting group was led by Gerald Selbee, a retiree in his seventies with a bachelor’s degree in math.

 

TWELVE: MISS MORE PLANES!

Assume that if you arrive at the airport 2 hours early, you have a 2% chance of missing the plane; if you arrive 1.5 hours early, you have a 5% chance of missing the plane; and if you arrive 1 hour early, you have a 15% chance of missing the plane.  Then, if you think of missing a plane as costing six hours of time, you can calculate the expected cost in utils of arriving at the airport 2 hours, 1.5 hours, and 1 hour early.

Option 1 –2 + 2% x (–6) = –2.12 utils
Option 2 –1.5 + 5% x (–6) = –1.8 utils
Option 3 –1 + 15% x (–6) = –1.9 utils

Under these assumptions, you should arrive at the airport 1.5 hours early.  Of course, maybe you really hate missing a plane.  Perhaps the cost of missing a plane is –20 utils.  In that case, if you redo the expected value, then Option 1 works out to be your best choice.

Ellenberg discusses Pascal’s wager.  Say that living a life of piety costs 100 utils.  But if the God of Christianity is real, and if you believe it and live accordingly, then the payoff is infinite joy.  Say there’s a 5% chance that the God of Christianity is real.  Then Pascal’s wager can be written:

(5%) x infinity + (95%) x (–100) = infinity

Illustration by Mariia Domnikova

No matter how small the odds of the Christian God’s existence, a tiny number times infinity is still infinity.  Ellenberg comments:

Pascal’s argument has serious flaws.  The gravest is that it suffers from… failing to consider all possible hypotheses.  In Pascal’s setup, there are only two options: that the God of Christianity is real and will reward that particular sector of the faithful, or that God doesn’t exist.  But what if there’s a God who damns Christians eternally?  Such a God is surely possible too, and this possibility alone suffices to kill the argument…

Utils can be useful for problems that don’t have well-defined dollar values.  But they can also be useful for problems that are stated in dollar values.  In 1738, Daniel Bernoulli put forward the St. Petersburg Paradox: “Peter tosses a coin and continues to do so until it should land ‘heads’ when it comes to the ground.  He agrees to give Paul one ducat if he gets ‘heads’ on the very first throw, two ducats if he gets it on the second, four if on the third, eight if on the fourth, and so on, so that with each additional throw the number of ducats he must pay is doubled.”  The question is: How much should Paul pay in order to play this game?  Paul’s expected return is:

(1/2) x 1 + (1/4) x 2 + (1/8) x 4 + (1/16) x 8 + (1/32) x 16 + …

This can be written as:

(1/2) + (1/2) + (1/2) + (1/2) + (1/2) + …

It would seem that the expected dollar value of playing the game is infinite, so Paul should be willing to spend any number of ducats in order to play.  Ellenberg:

The mistake, Bernoulli said, is to say that a ducat is a ducat is a ducat… having two thousand ducats isn’t twice as good as having one thousand; it is less than twice as good, because a thousand ducats is worth less to a person who already has a thousand ducats than it is to the person who has none…

Bernoulli thought that utility grew like the logarithm, so that the kth prize of 2k ducats was worth just k utils…

In Bernoulli’s formulation, the expected utility of the St. Petersburg game is the sum

(1/2) x 1 + (1/4) x 2 + (1/8) x 3 + (1/16) x 4 + ….

This series equals 2.  To see this, consider:

(1/2) + (1/4) + (1/8) + (1/16) + (1/32) + … = 1

    (1/4) + (1/8) + (1/16) + (1/32) + … = 1/2

        (1/8) + (1/16) + (1/32) + … = 1/4

            (1/16) + (1/32) + … = 1/8

                  (1/32) + … = 1/16

The top row is Zeno’s paradox.  The series converges to 1.  The second row is just the top row, but without the 1/2 at the beginning, so it must equal 1/2.  The third row is the same as the second, but without the 1/4 at the beginning, so it must equal 1/4.  And so forth.  If you look at the series of what each row equals, you get:

1 + (1/2) + (1/4) + (1/8) + (1/16) + (1/32) + … = 2

If you sum up the columns of the five rows above, you get:

1/2 + 2/4 + 3/8 + 4/16 + 5/32 + … = 2

This series converges to 2.   So the expected utility of the St. Petersburg game is 2.  Paul should be willing to pay 2 ducats in order to play this game.  2 is substantially less than infinity.

 

THIRTEEN: WHERE THE TRAIN TRACKS MEET

Ellenberg writes:

Mathematical elegance and practical utility are close companions, as the history of science has shown again and again.  Sometimes scientists discover the theory and leave it to mathematicians to figure out why it’s elegant, and other times mathematicians develop an elegant theory and leave it to scientists to figure out what it’s good for.

Ellenberg introduces the projective plane, which is governed by two axioms:

    • Every pair of points is contained in exactly one common line.
    • Every pair of lines contains exactly one common point.

If you trace out two parallel lines traveling away from you, those two lines meet on the horizon.  Call that point P.  But that vanishing point is defined to be the same point P if you turned around and looked the opposite direction.  In this way, if you think about a graph, the y-axis is a circle going vertically, while the x-axis is a circle going horizontally.

Are there other geometries besides the projective plane that satisfy the two axioms?  Yes.  For example, the Fano plane:

 

Ellenberg comments:

For Fano and his intellectual heirs, it doesn’t matter whether a line “looks like” a line, a circle, a mallard duck, or anything else—all that matters is that lines obey the laws of lines, set down by Euclid and his successors.  If it walks like geometry, and it quacks like geometry, we call it geometry.  To one way of thinking, this move constitutes a rupture between mathematics and reality, and is to be resisted.  But that view is too conservative.  The bold idea that we can think geometrically about systems that don’t look like Euclidean space, and even call these systems “geometries” with head held high, turned out to be critical to understanding the geometry of relativistic space-time we live in; and nowadays we use generalized geometric ideas to map Internet landscapes, which are even further removed from anything Euclid would recognize.  That’s part of the glory of math; we develop a body of ideas, and once they’re correct, they’re correct, even when applied far, far outside the context in which they were first conceived.

Looking at Fano’s plane again, there are seven lines (one of which is the circle), each of which has three points on it.

    • 124
    • 135
    • 167
    • 257
    • 347
    • 236
    • 456

Look familiar?  Ellenberg:

This is none other than the seven-ticket combo we saw in the last section, the one that hits each pair of numbers exactly once, guaranteeing a minimum payoff…

…it’s simply geometry.  Each pair of numbers appears on exactly one ticket, because each pair of points appears on exactly one line.  It’s just Euclid, even though we’re speaking now of points and lines Euclid would not have recognized as such.

What about the Massachusetts lottery?  There is no geometry that fits the precise requirements.  Ellenberg says we should consider the theory of digital signal processing.  Say we’re trying to send the following digital message:

1110101…

Assume this is a communication to a satellite that says, “Turn on right thruster.”  But what if the message get garbled and sends the following instead:

1010101…

This could mean, “Turn on the left thruster.”  That would be a serious problem.  A solution is to send each digit twice.  So the original message looks like:

11 11 11 00 11 00 11…

However, this still doesn’t solve the issue of potential garbling because if 11 gets turned into 01, we don’t know if 01 is supposed to be 11 or 00.  This can be solved by repeating each digit three times.  So the message looks like:

111 111 111 000 111 000 111…

Now if 111 gets garbled and turns into 101, the satellite knows it’s supposed to be 111.  That’s not a guarantee, of course, but there’s at least a high probability that the original message was 111.  Ellenberg asks whether there can really be a mathematical theory of communication.  Ellenberg comments:

Understand this: I warmly endorse, in fact highly recommend, a bristly skepticism in the face of all claims that such-and-such an entity can be explained, or tamed, or fully understood, by mathematical means.

And yet the history of mathematics is a history of aggressive territorial expansion, as mathematical techniques get broader and richer, and mathematicians find ways to address questions previously thought of as outside their domain.  “A mathematical theory of probability” sounds unexceptional now, but once it would have seemed a massive overreach; math was about the certain and the true, not the random and the maybe-so!  All that changed when Pascal, Bernoulli, and others found mathematical laws that governed the workings of chance.  A mathematical theory of infinity?  Before the work of Georg Cantor in the nineteenth century, the study of the infinite was as much theology as science; now we understand Cantor’s theory of multiple infinities, each one infinitely larger than the last, well enough to teach it to first-year math majors.

These mathematical formalisms don’t capture every detail of the phenomena they describe, and aren’t intended to.  There are questions about randomness, for instance, about which probability theory is silent…

People are working on mathematical theories that can explain and describe consciousness, society, aesthetics, and other areas.  Though success has thus far been limited, mathematics may end up getting some important points right, notes Ellenberg.

Regarding information theory, a colleague of Claude Shannon, Richard Hamming, was trying to run his programs on the weekend, but any error would halt the computation, with no one to get the machine running again until Monday morning.  So Hamming thought of a way for the machine to correct its own errors.  First he broke the message into blocks of three symbols.  Then he invented a way for each three-digit block to be transformed into a seven-digit string.  This is the Hamming code:

    • 000 -> 0000000
    • 001 -> 0010111
    • 010 -> 0101011
    • 011 -> 0111100
    • 101 -> 1011010
    • 110 -> 1100110
    • 100 -> 1001101
    • 111 -> 1110001

If the receiver gets anything that isn’t a code word, something has gone wrong.  Also, if a message is only different from a code word by one digit, you can safely infer which code word was intended.

Look back at the lines in the Fano plane.

…the seven nonzero code words in the Hamming code match up exactly to the seven lines in the Fano plane [124 is 0010111, for instance, while 135 is 0101011].  The Hamming code and the Fano plane… are exactly the same mathematical object in two different outfits!

Note that the Hamming code sends just seven bits for every three bits of your original message, a more efficient ratio of 2.33 (vs. 3 for the repeat-three-times code presented earlier).  There’s also the idea of Hamming distance, which is the number of bits you need to alter to go from one code word to another.  Two different code words are at a Hamming distance of at least 4 from each other.

Ellenberg continues:

Hamming’s notion of “distance” follows Fano’s philosophy… But why stop there?  The set of points at distance less than or equal to 1 from a given central point… is called a circle, or, if we are in higher dimensions, a sphere.  So we’re compelled to call the set of strings at Hamming distance at most 1 from a code word a “Hamming sphere,” with the code word at the center.  For a code to be an error-correcting code, no string—no point, if we’re to take this geometric analogy seriously—can be within distance 1 of two different code words; in other words, we ask that no two of the Hamming spheres centered at the code words share any points.

So the problem of constructing error-correcting codes has the same structure as a classical geometric problem, that of sphere packing: how do we fit a bunch of equal-sized spheres as tightly as possible into a small space, in such a way that no two spheres overlap?  More succinctly, how many oranges can you stuff into a box?

If playing the lottery is fun, then it’s OK to spend a few dollars regularly playing.  Also, you won’t end up in a lower class by playing, but if you get lucky and win, you could end up in a higher class.  The bottom line, as Ellenberg says, is that mathematics gives you permission to go ahead and play Powerball if it’s fun for you.

Ellenberg concludes by comparing an entrepreneur to someone who plays the lottery:

…That’s the nature of entrepreneurship: you balance a very, very small probability of making a fortune against a modest probability of eking out a living against a substantially larger probability of losing your pile, and for a large proportion of potential entrepreneurs, when you crunch the numbers, the expected financial value, like that of a lottery ticket, is less than zero… And yet society benefits from a world in which people, against their wiser judgment, launch businesses.

Perhaps the biggest part of the utility of running a business, notes Ellenberg, is the act of realizing a dream, or even trying to realize it.

 

PART IV: REGRESSION

FOURTEEN: THE TRIUMPH OF MEDIOCRITY

Horace Secrist was a professor of statistics and director of the Bureau for Business Research at Northwestern.  Since 1920 and into the great crash, Secrist had been tracking a number of statistics on various businesses.  What he found was reversion to the mean, i.e., regression to mediocrity:

Secrist found the same phenomenon in every kind of business.  Hardware stores regressed to mediocrity; so did grocery stores.  And it didn’t matter what metric you used.  Secrist tried measuring his companies by the ratio of wages to sales, the ratio of rent to sales, and whatever other economic stat he could put his hands on.  It didn’t matter.  With time, the top performers started to look and behave just like the members of the common mass.

Similarly, Secrist found that the bottom performers improved and became more like the average.  Secrist’s views descend from those of the nineteenth-century British scientist, Francis Galton.

Galton had a peripatetic education; he tried studying mathematics at Cambridge but was defeated by the brutal Tripos exam, and devoted himself intermittently to the study of medicine, the line of work his parents had planned for him.  But after his father died in 1844, leaving him a substantial fortune, he found himself suddenly less motivated to pursue a traditional career.  For a while Galton was an explorer, leading expeditions into the African interior.  But the epochal publication of The Origin of Species in 1859 catalyzed a drastic shift in his interests: …from then on, the greater share of Galton’s work was devoted to the heredity of human characteristics, both physical and mental.

(Sir Francis Galton in the 1850s or early 1860s, scanned from Karl Pearson’s biography, via Wikimedia Commons)

Galton discovered regression to the mean.  For example, tall parents were likely to have tall children, but those children were usually not as tall as the parents.  Similarly, short parents were likely to have short children, but those children were usually not as share as the parents.  In both cases, there is regression to the mean.  Ellenberg:

So, too, Galton reasoned, must it be for mental achievement.  And this conforms with common experience; the children of a great composer, or scientist, or political leader, often excel in the same field, but seldom so much so as their illustrious parent.  Galton was observing the same phenomenon that Secrist would uncover in the operations of business.  Excellence doesn’t persist; time passes, and mediocrity asserts itself.

People are tall due to a combination of genetics and chance.  Genetics persist, but chance does not persist.  That’s why the children of tall parents tend to be tall, but not as tall as their parents: the factor of chance does not persist.  Ellenberg explains:

It’s just the same for businesses.  Secrist wasn’t wrong about the firms that had the fattest profits in 1922; it’s likely that they ranked among the most well-managed companies in their sectors.  But they were lucky, too.  As time went by, their management might well have remained superior in wisdom and judgment.  But the companies that were lucky in 1922 were no more likely than any other companies to be lucky ten years later.

 

FIFTEEN: GALTON’S ELLIPSE

A scatterplot of the heights of fathers versus the heights of sons arranges itself in an elliptical pattern, where the heights of sons are closer to the mean than the heights of fathers.  Ellenberg:

…we have ellipses of various levels of skinniness.  That skinniness, which the classical geometers call the eccentricity of the ellipse, is a measure of the extent to which the height of the father determines that of the son.  High eccentricity means that heredity is powerful and regression to the mean is weak; low eccentricity means the opposite, that regression to the mean holds sway.  Galton called his measure correlation, the term we still use today.  If Galton’s ellipse is almost round, the correlation is near 0; when the ellipse is skinny, lined up along the northeast-southwest axis, the correlation comes close to 1.

It’s important to note that correlation is not transitive.  A may be correlated with B, and B with C, but that doesn’t mean A is correlated with C.  Ellenberg:

If correlation were transitive, medical research would be a lot easier than it actually is.  Decades of observation and data collection have given us lots of known correlations to work with.  If we had transitivity, doctors could just chain these together into reliable interventions.  We know that women’s estrogen levels are correlated with lower risk of heart disease, and we know that hormone replacement therapy can raise those levels, so you might expect hormone replacement therapy to be protective against heart disease.  And, indeed, that used to be conventional clinical wisdom.  But the truth, as you’ve probably heard, is a lot more complicated.  In the early 2000s, the Women’s Health Initiative, a long-term study involving a gigantic randomized clinical trial, reported that hormone replacement therapy with estrogen and progestin appeared actually to increase the risk of heart disease in the population they studied…

In the real world, it’s next to impossible to predict what effect a drug will have on a disease, even if you know a lot about how it affects biomarkers like HDL or estrogen level.  The human body is an immensely complex system, and there are only a few of its features we can measure, let alone manipulate.  But on the correlations we can observe, there are lots of drugs that might plausibly have a desired health effect.  And so you try them out in experiments, and most of them fail dismally.  To work in drug development requires a resilient psyche, not to mention a vast pool of capital.

 

SIXTEEN: DOES LUNG CANCER MAKE YOU SMOKE CIGARETTES?

Correlation is not causation:

(Photo by Alain Lacroix)

Ellenberg writes:

Teasing apart correlations that come from causal relationships from those that don’t is a maddeningly hard problem, even in cases you might think of as obvious, like the relation between smoking and lung cancer.  At the turn of the twentieth century, lung cancer was an extremely rare disease.  But by 1947, the disease accounted for nearly a fifth of cancer deaths among British men, killing fifteen times as many people as it had a few decades earlier.  At first, many researchers thought that lung cancer was simply being diagnosed more effectively than before, but it soon became clear that the increase in cases was too big and too fast to be accounted for by any such effect.  Lung cancer really was on the rise.  But no one was sure what to blame.  Maybe it was smoke from factories, maybe increased levels of car exhaust, or maybe some substance not even thought of as a pollutant.  Or maybe it was cigarette smoking, whose popularity had exploded during the same period.

In the 1950s, there were some large studies that showed a correlation between smoking and lung cancer.  That doesn’t establish causality.  Perhaps lung cancer causes smoking, or perhaps there is a common cause that leads to both smoking and lung cancer.  It’s not very reasonable to assert that lung cancer causes smoking, because the tumor would have to reach back years into the past to start causing smoking.  But it is possible that there is some cause responsible for both smoking and lung cancer.  Keep in mind, notes Ellenberg, that in the 1950s, no chemical component of tobacco had yet been shown to produce tumors in the lab.  Today, things are different.  We know that smoking does cause lung cancer:

We know a lot more now about cancer and how tobacco brings it about.  That smoking gives you cancer is no longer in serious dispute.

Back in the 1950s, the evidence was not so clear.  But this seemed to change in the 1960s.  Ellenberg:

By 1964, the association between smoking and cancer had appeared consistently across study after study.  Heavier smokers suffered more cancer than lighter smokers, and cancer was most likely at the point of contact between tobacco and human tissue; cigarette smokers got more lung cancer, pipe smokers more lip cancer.  Ex-smokers were less prone to cancer than smokers who kept up the habit.  All these factors combined to lead the surgeon general’s committee to the conclusion that smoking was not just correlated with lung cancer, but caused lung cancer, and that efforts to reduce tobacco consumption would be likely to lengthen American lives.

From a public policy point of view, whether to come out against smoking depends on the expected value of doing so:

…So we don’t and can’t know the exact expected value of launching a campaign against…tobacco.  But often we can say with confidence that the expected value is positive.  Again, that doesn’t mean the campaign is sure to have good effects, only that the sum total of all similar campaigns, over time, is likely to do more good than harm.  The very nature of uncertainty is that we don’t know which of our choices will help, like attacking tobacco, and which will hurt, like recommending hormone replacement therapy.  But one thing’s for certain: refraining from making recommendations at all, on the grounds that they might be wrong, is a losing strategy.  It’s a lot like George Stigler’s advice about missing planes.  If you never give advice until you’re sure it’s right, you’re not giving enough advice.

 

PART V: EXISTENCE

SEVENTEEN: THERE IS NO SUCH THING AS PUBLIC OPINION

Americans report that they would rather cut government programs—and make government smaller—than pay more taxes.  Ellenberg:

But which government programs?  That’s where things get sticky.  It turns out the things the U.S. government spends money on are things people kind of like.  A Pew Research poll from February 2011 asked Americans about thirteen categories of government spending: in eleven of those categories, deficit or no deficit, more people wanted to increase spending than dial it down.  Only foreign aid and unemployment insurance—which, combined, accounted for under 5% of 2010 spending—got the ax.  That, too, agrees with years of data; the average American is always eager to slash foreign aid, occasionally tolerant of cuts of welfare or defense, and pretty gung ho for increased spending on every single other program our taxes fund.

Oh, yeah, and we want small government.

(Photo by David Watmough)

Ellenberg again:

The average American thinks there are plenty of non-worthwhile federal programs that are wasting our money and is ready and willing to put them on the chopping block to make ends meet.  The problem is, there’s no consensus on which programs are the worthless ones.  In large part, that’s because most Americans think the programs that benefit them personally are the ones that must, at all costs, be preserved…

Ellenberg continues:

The “majority rules” system is simple and elegant and feels fair, but it’s at its best when deciding between just two options.  Any more than two, and contradictions start to seep into the majority’s preferences.  As I write this, Americans are sharply divided over President Obama’s signature domestic policy accomplishment, the Affordable Care Act.  In an October 2010 poll of likely voters, 52% of respondents said they opposed the law, while only 41% supported it.  Bad news for Obama?  Not once you break down the numbers.  Outright repeal of health care reform was favored by 37%, with another 10% saying the law should be weakened; but 15% preferred to leave it as is, and 36% said the ACA should be expanded to change the current health care system more than it currently does.  That suggests that many of the law’s opponents are to Obama’s left, not his right.  There are (at least) three choices here: leave the health care law alone, kill it, or make it stronger.  And each of the three choices is opposed by most Americans.

Ellenberg adds:

The incoherence of the majority creates plentiful opportunities to mislead.  Here’s how Fox News might report the poll results above:

Majority of Americans oppose Obamacare!

And this is how it might look to MSNBC:

Majority of Americans want to preserve or strengthen Obamacare!

These two headlines tell very different stories about public opinion.  Annoyingly enough, both are true.

But both are incomplete.  The poll watcher who aspires not to be wrong has to test each of the poll’s options, to see whether it might break down into different-colored pieces.  Fifty-six percent of the population disapproves of President Obama’s policy in the Middle East?  That impressive figure might include people from both the no-blood-for-oil left and the nuke-‘em-all right, with a few Pat Buchananists and devoted libertarians in the mix.  By itself, it tells us just about nothing about what the people really want.

Ellenberg later writes about experiments studying the decision making of slime mold.  Note that slime mold likes to eat oats and likes to avoid light.  (Note also that you can train a slime mold to navigate through a maze by using oats.)  In one experiment, the slime mold is faced with a choice: 3 grams of oats in the dark (3-dark) versus 5 grams of oats in the light (5-light).  In this scenario, the slime mold picks 3-dark half the time and 5-light half the time.

Now, if you replace the 5 grams of oats with 10 grams (10-light), the slime mold chooses 10-light every time.

However, something strange happens if you have 3-dark and 5-light options, but then add a third option: 1-dark (1 gram of oats in the dark).  You might think that the slime mold would never pick 1-dark.  That’s true.  But you might also think that the slime mold continues to pick 3-dark half the time and 5-light half the time.  But that’s not what the slime mold does.  Faced with 1-dark, 3-dark, and 5-light, the slime mold picks 3-dark more than three times as often as 5-light.  Ellenberg:

The mathematical buzzword in play here is “independence of irrelevant alternatives.”  That’s a rule that says, whether you’re a slime mold, a human being, or a democratic nation, if you have a choice between two options, A and B, the presence of a third option, C, shouldn’t affect which of A and B you like better.

Ellenberg gives an example.  In Florida in the 2000 election, the majority preferred Gore over Bush.  However, the presence of an irrelevant alternative—Ralph Nader—tipped the election to Bush.  Bush got 48.85% of the vote, Gore got 48.84%, while Nader got 1.6%.  Back to the slime mold:

…the slime mold likes the small, unlit pile of oats about as much as it likes the big, brightly lit one.  But if you introduce a really small unlit pile of oats, the small dark pile looks better by comparison; so much so that the slime mold decides to choose it over the big bright pile almost all the time.

This phenomenon is called the “asymmetric domination effect,” and slime molds are not the only creatures subject to it.  Biologists have found jays, honeybees, and hummingbirds acting in the same seemingly irrational way.

Not to mention humans!

Ellenberg tries to explain human irrationality:

Maybe individual people seem irrational because they aren’t really individuals!  Each one of us is a little nation-state, doing our best to settle disputes and broker compromises between the squabbling voices that drive us.  The results don’t always make sense.  But they somehow allow us, like the slime molds, to shamble along without making too many terrible mistakes.  Democracy is a mess—but it kind of works.

 

EIGHTEEN: “OUT OF NOTHING I HAVE CREATED A STRANGE NEW UNIVERSE”

Ellenberg poses an interesting question:

Are we trying to figure out what’s true, or are we trying to figure out what conclusions are licensed by our rules and procedures?  Hopefully the two notions frequently agree; but all the difficulty, and all the conceptually interesting stuff, happens at the points where they diverge.

You might think it’s obvious that figuring out what’s true is always our proper business.  But that’s not always the case in criminal law, where the difference presents itself quite starkly in the form of defendants who committed the crime but who cannot be convicted (say, because evidence was obtained improperly) or who are innocent of the crime but are convicted anyway.  What’s justice here—to punish the guilty and free the innocent, or to follow criminal procedure wherever it leads us?  In experimental science, we’ve already seen the dispute with R.A. Fisher on one side and Jerzy Neyman and Egon Pearson on the other.  Are we, as Fisher thought, trying to figure out which hypotheses we should actually believe are true?  Or are we to follow the Neyman-Pearson philosophy, under which we resist thinking about the truth of hypotheses at all and merely ask: Which hypotheses are we to certify as correct, whether they’re really true or not, according to our chosen rules of inference?

You might think that mathematics itself doesn’t have such problems.  But it does.  Consider the parallel postulate, Euclid’s fifth axiom:  “If P is a point and L is a line not passing through P, there is exactly one line through P parallel to L.”  Even Euclid was thought to dislike his fifth axiom.  He proved the first twenty-eight propositions in the Elements using only the first four axioms.

In 1820, the Hungarian noble Farkas Bolyai, who had given years of his life to the problem, wrote a letter to his son trying to dissuade him from attempting to solve the same problem.  However, Janos Bolyai ignored his father’s advice.  By 1823, he had an outline of the solution.  He wrote to his father, “out of nothing I have created a strange new universe.”

Jonas Bolyai inverted the problem and asked: If the parallel axiom were false, would a contradiction follow?  Bolyai realized the answer was no:

…there was another geometry, not Euclid’s but something else, in which the first four axioms were correct but the parallel postulate was not.  Thus, there can be no proof of the parallel postulate from the other axioms; such a proof would rule out the possibility of Bolyai’s geometry.  But there it was.

Sometimes, a mathematical development is “in the air”—for reasons only poorly understood, the community is ready for a certain advance to come, and it comes from several sources at once.  Just as Bolyai was constructing his non-Euclidean geometry in Austria-Hungary, Nikolai Lobachevskii was doing the same in Russia.  And the great Carl Friedrich Gauss, an old friend of the senior Bolyai, had formulated many of the same ideas in work that had not yet seen print…

A few decades later, Bernhard Riemann pointed out that there’s a simpler non-Euclidean geometry: the geometry of the sphere.  Consider Euclid’s first four axioms:

    • There is a Line joining any two Points.
    • Any Line segment can be extended to a Line segment of any desired length.
    • For every Line segment L, there is a Circle which has L as a radius.
    • All Right Angles are congruent to each other.

Ellenberg on Riemann’s spherical geometry:

A Point is a pair of points on the sphere which are antipodal, or diametrically opposite each other.  A Line is a “great circle”—that is, a circle on the sphere’s surface—and a Line segment is a segment of such a circle.  A Circle is a circle, now allowed to be of any size.

Using these definitions, Euclid’s first four axioms are true.  Ellenberg comments:

Here’s the thing; once you understand that the first four axioms apply to many different geometries, then any theorem Euclid proves from only those axioms must be true, not only in Euclid’s geometry, but in all the geometries where those axioms hold.  It’s a kind of mathematical force multiplier; from one proof, you get many theorems.

And these theorems are not just about abstract geometries made up to prove a point.  Post-Einstein, we understand that non-Euclidean geometry is not just a game; like it or not, it’s the way space-time actually looks.

This is a story told in mathematics again and again: we develop a method that works for one problem, and if it is a good method, one that really contains a new idea, we typically find that the same proof works in many different contexts, which may be as different from the original as a sphere is from a plane, or more so.

Ellenberg continues:

The tradition is called “formalism.”  It’s what G. H. Hardy was talking about when he remarked, admiringly, that mathematicians of the nineteenth century finally began to ask what things like

1 – 1 + 1 – 1 + …

should be defined to be, rather than what they were… In the purest version of this view, mathematics becomes a kind of game played with symbols and words.  A statement is a theorem precisely if it follows by logical steps from the axioms.  But what the axioms and theorems refer to, what they mean, is up for grabs.  What is a Point, or a Line…?  It can be anything that behaves the way the axioms demand, and the meaning we should choose is whichever one suits our present needs.  A purely formal geometry is a geometry you can in principle do without ever having seen or imagined a point or a line; it is a geometry in which it’s irrelevant what points and lines, understood in the usual way, are actually like.

Mathematical formalism shares similarities with legal formalism.

In Scalia’s view, when judges try to understand what the law intends—its spirit—they’re inevitably bamboozled by their own prejudices and desires.  Better to stick to the words of the Constitution and the statutes, treating them as axioms from which judgments can be derived by something like logical deduction.

Ellenberg continues:

Formalism has an austere elegance.  It appeals to people like G. H. Hardy, Antonin Scalia, and me, who relish that feeling of a nice rigid theory shut tight against contradiction.  But it’s not easy to hold to principles like this consistently, and it’s not clear it’s even wise.  Even Justice Scalia has occasionally conceded that when the literal words of the law seem to require an absurd judgment, the literal words have to be set aside in favor of a reasonable guess as to what Congress must have meant.  In just the same way, no scientist really wants to be bound strictly by the rules of significance, no matter what they say their principles are.  When you run two experiments, one testing a clinical treatment that seems theoretically promising and the other testing whether dead salmon respond emotionally to romantic photos, and both experiments succeed with p-values of .03, you don’t really want to treat the two hypotheses the same.

Ellenberg then notes that the German mathematician David Hilbert was formalism’s greatest champion in mathematics.

(David Hilbert before 1912, via Wikimedia Commons)

Hilbert wanted to create a purely formal mathematics.  To say that a statement was true was to say that it could be derived logically from the axioms.  However, says Ellenberg:

Mathematics has a nasty habit of showing that, sometimes, what’s obviously true is absolutely wrong.

For an example, consider set theory.  An ouroboric set has itself as a member.

Let NO be the set of all non-ouroboric sets…

Is NO ouroboric or not?  That is, is NO an element of NO?  By definition, if NO is ouroboric, then NO cannot be in NO, which consists only of non-ouroboric sets.  But to say NO is not an element of NO is precisely to say NO is non-ouroboric; it does not contain itself.

But wait a minute—if NO is non-ouroboric, then it is an element of NO, which is the set of all non-ouroboric sets.  Now NO is an element of NO after all, which is to say that NO is ouroboric.

If NO is ouroboric, it isn’t, and if it isn’t, it is.

But could finite arithmetic be proved consistent?  Ellenberg:

Hilbert sought a finitary proof of consistency, one that did not make reference to any infinite sets, one that a rational mind couldn’t help but wholly believe.

But Hilbert was to be disappointed.  In 1931, Kurt Godel proved in his famous second incompleteness theorem that there could be no finite proof of the consistency of arithmetic.  He had killed Hilbert’s program with a single stroke.

Nonetheless:

Hilbert’s style of mathematics survived the death of his formalist program.

 

HOW TO BE RIGHT

Nearly everyone has heard of Theodore Roosevelt’s speech “Citizenship in a Republic,” which he delivered in Paris in 1910.  Here’s the part people love to quote:

It is not the critic who counts; not the man who points out how the strong man stumbles, or where the doer of deeds could have done them better.  The credit belongs to the man who is actually in the arena, whose faced is marred by dust and sweat and blood; who strives valiantly; who errs, who comes short again and again, because there is no effort without error and shortcoming; but who does actually strive to do the deeds; who knows great enthusiasms, the great devotions; who spends himself in a worthy cause; who at the best knows in the end the triumph of high achievement, and who at the worst, if he fails, at least fails while daring greatly, so that his place shall never be with those cold and timid souls who neither know victory nor defeat.

Ellenberg writes:

And yet, when Roosevelt says, “The closet philosopher, the refined and cultured individual who from his library tells how men ought to be governed under ideal conditions, is of no use in actual governmental work,” I think of Condorcet, who spent his time in the library doing just that, and who contributed more to the French state than most of his time’s more practical men.  And when Roosevelt sneers at the cold and timid souls who sit on the sidelines and second-guess the warriors, I come back to Abraham Wald, who as far as I know went his whole life without lifting a weapon in anger, but who nonetheless played a serious part in the American war effort, precisely by counseling the doers of deeds how to do them better.  He was unsweaty, undusty, and unbloody, but he was right.  He was a critic who counted.

Mathematics not only deals with certainties, but also allows us to deal with uncertainty.  Ellenberg:

Math gives us a way of being unsure in a principled way: not just throwing up our hands and saying “huh,” but rather making a firm assertion: “I’m not sure, this is why I’m not sure, and this is roughly how not-sure I am.”  Or even more: “I’m unsure, and you should be too.”

Ellenberg comments:

The paladin of principled uncertainty in our time is Nate Silver, the online-poker-player-turned-baseball-statistics-maven-turned-political analyst…

What made Silver so good?  In large part, it’s that he was willing to talk about uncertainty, willing to treat uncertainty not as a sign of weakness but as a real thing in the world, a thing that can be studied with scientific rigor and employed to good effect.  If it’s September 2012 and you ask a bunch of political pundits, “Who’s going to be elected president in November?” a bunch of them are going to say, “Obama is,” and a somewhat smaller bunch are going to say, “Romney is,” and the point is that all of those people are wrong, because the right answer is the kind of answer that Silver, almost alone in the broad-reach media, was willing to give: “Either one might win, but Obama is substantially more likely to win.”

One important piece of advice for mathematicians who are trying to prove a theorem is to divide your time between trying to prove the theorem and trying to disprove it.  First, the theorem could be wrong, in which case the sooner you realize that, the better.  Second, if the theorem is true and you try to disprove it, eventually you will get a better idea of how to prove that the theorem is true.

This self-critical attitude applies to other areas besides mathematics, says Ellenberg:

Proving by day and disproving by might is not just for mathematics.  I find it’s a good habit to put pressure on all your beliefs, social, political, scientific, and philosophical.  Believe whatever you believe by day; but at night, argue against the propositions you hold most dear.  Don’t cheat!  To the greatest extent possible you have to think as though you believe what you don’t believe.  And if you can’t talk yourself out of your existing beliefs, you’ll know a lot more about why you believe what you believe.  You’ll have come a little closer to a proof.

Ellenberg concludes:

What’s true is that the sensation of mathematical understanding—of suddenly knowing what’s going on, with total certainty, all the way to the bottom—is a special thing, attainable in few if any other places in life.  You feel you’ve reached in to the universe’s guts and put your hand on the wire.  It’s hard to describe to people who haven’t experienced it.

We are not free to say whatever we like about the wild entities we make up.  They require definition, and having been defined, they are no more psychedelic than trees and fish; they are what they are.  To do mathematics is to be, at once, touched by fire and bound by reason.  This is no contradiction.  Logic forms a narrow channel through which intuition flows with vastly augmented force.

The lessons of mathematics are simple ones and there are no numbers in them: that there is structure in the world; that we can hope to understand some of it and not just gape at what our senses present to us; that our intuition is stronger with a formal exoskeleton than without one.  And that mathematical certainty is one thing, the softer convictions we find attached to us in everyday life another, and we should keep track of the difference if we can.

Every time you observe that more of a good thing is not always better; or you remember that improbable things happen a lot, given enough chances, and resist the lure of the Baltimore stockbroker; or you make a decision based not just on the most likely future, but on the cloud of all possible futures, with attention to which ones are likely and which ones are not; or you let go of the idea that the beliefs of groups should be subject to the same rules as beliefs of individuals; or, simply, you find that cognitive sweet spot where you can let your intuition run wild on the network of tracks formal reasoning makes for it; without writing down an equation or drawing a graph, you are doing mathematics, the extension of common sense by other means.  When are you going to use it?  You’ve been using mathematics since you were born and you’ll probably never stop.  Use it well.

 

BOOLE MICROCAP FUND

An equal weighted group of micro caps generally far outperforms an equal weighted (or cap-weighted) group of larger stocks over time.  See the historical chart here:  https://boolefund.com/best-performers-microcap-stocks/

This outperformance increases significantly by focusing on cheap micro caps.  Performance can be further boosted by isolating cheap microcap companies that show improving fundamentals.  We rank microcap stocks based on these and similar criteria.

There are roughly 10-20 positions in the portfolio.  The size of each position is determined by its rank.  Typically the largest position is 15-20% (at cost), while the average position is 8-10% (at cost).  Positions are held for 3 to 5 years unless a stock approaches intrinsic value sooner or an error has been discovered.

The mission of the Boole Fund is to outperform the S&P 500 Index by at least 5% per year (net of fees) over 5-year periods.  We also aim to outpace the Russell Microcap Index by at least 2% per year (net).  The Boole Fund has low fees.

 

If you are interested in finding out more, please e-mail me or leave a comment.

My e-mail: jb@boolefund.com

 

 

 

Disclosures: Past performance is not a guarantee or a reliable indicator of future results. All investments contain risk and may lose value. This material is distributed for informational purposes only. Forecasts, estimates, and certain information contained herein should not be considered as investment advice or a recommendation of any particular security, strategy or investment product. Information contained herein has been obtained from sources believed to be reliable, but not guaranteed. No part of this article may be reproduced in any form, or referred to in any other publication, without express written permission of Boole Capital, LLC.