November 01, 2015
Becoming a Superforecaster, Gold, and Feedback Effects
On August 13, 2007, the Financial Times reported that two large hedge funds managed by Goldman Sachs had both lost over a quarter of their value in a week, requiring the injection of $3 billion to support them. Goldman Sachs ’ CFO David Viniar stated of the market price gyrations that caused these losses, “We were seeing things that were 25 standard deviation moves, several days in a row.” If we assume that market price changes are normally distributed, which apparently Viniar’s risk models did (and is common practice), then a twenty-five standard deviation move, according to pundits, implies odds comparable to winning the UK lottery 21 times in a row or finding a 30-foot tall human.All through time, people have basically acted and reacted the same way in the market as a result of: greed, fear, ignorance, and hope. That is why the numerical formations and patterns recur on a constant basis.
~ Jesse Livermore, How to Trade in Stocks, 1940
To explain the reliance of financial forecasters on incomplete data and poor models, Nobel Laureate Robert Shiller noted that “Theorists like models with order, harmony and beauty” while “Academics like ideas that will lead to econometric studies.” By contrast, behavioral economists speak of the influence of human psychology, and this focus on the softer side can be off-putting to economists. According to Shiller: “People in ambiguous situations will focus on the person who has the most coherent model.” This search for coherence often misses the interesting nooks and crannies of financial market behavior, where some actual predictability may lurk.
Fortunately the science of individual forecasting made a dramatic advance with the release of psychologist Philip Tetlock's excellent new book - Superforecasting. Today's newsletter will discuss some lessons for improving forecasting accuracy. But applying these individual lessons to markets requires a nuanced understanding of how markets set prices. Feedback effects (George Soros' Reflexivity) - how information, herd behavior, and prices interact over various time horizons - are a unique facet of financial markets, and an understanding of these effects is essential to good forecasting. To illustrate, we examine the recent Gold bubble and decline.We really can't forecast all that well, and yet we pretend that we can, but we really can't.
~ Alan Greenspan
Is It Possible to Forecast Markets?
Having access to more information, more quickly, is fundamental to optimal forecasting. Yet we are overwhelmed with such information. IBM estimates that global data volume doubles every 18 months. For example, as of 2013, over 100 billion business emails were sent daily. If computers could autonomously make sense of this data deluge, then the volume would not be problematic. But computers do not understand meaning, and they need human assistance to develop better forecasts from the data.Most of the empirical research in finance, whether published in academic journals or put into production as an active trading strategy by an investment manager, is likely false.
~ Campbell Harvey and Yan Liu, “Evaluating Trading Strategies,” 2014.
For the humans who create forecasting models on this data, making predictions is an exercise fraught with risk. The potential problems range across humans biases to data errors to inappropriate application of statistical tools. Perhaps the most common problem is overfitting - a problem made worse by the availability of increasingly complex statistical tools. MIT professor Andrew Lo summed up the problem of statistical overfitting neatly with this statement, "I’ve often said that, you know, whatever back test you’d like to see, I can certainly produce it for you. If you torture the data long enough, it will basically tell you anything you want."
Given the expertise needed to find predictive meaning in the massive volume of global information, and the predictably poor models and forecasts of experts, Phillip Tetlock set out to identify the traits of what he calls Superforecasters - people who are excellent at synthesizing information and making predictions of the future.
Superforecasting
Philip Tetlock's book "Superforecasting" is phenomenal. If you're reading this newsletter you're probably familiar with many of the ideas Tetlock cites in his book - Prospect Theory, probabilistic versus black-and-white judgment, the growth mindset, and the importance of empirical testing. Tetlock's synthesis of these concepts with his own ground-breaking research, and tales of institutional and expert intransigence, make for a compelling narrative.We will need to blend computer-based forecasting and subjective judgement in the future. So it's time to get serious about both.
~ Phillip Tetlock and Dan Gardner, Superforecasting, 2015.
Tetlock backs up his research with remarkable anecdotes. Several stories about the failures of intelligence estimates illustrate how innate cognitive biases can be very difficult to dislodge, especially in large institutions. In one example from the 1960s, the phrase "a strong possibility" was used in an intelligence estimate to describe the likely outcome of an important event. The authors agreed on the statement to convey their confidence in the forecast. Yet when asked separately to explain the numerical probability inherent in this estimate, the authors rated the probability as ranging between 20 and 80% likelihood, meaning the language was meaningless and subject to interpretation. Tetlock notes that President John F. Kennedy was was told the CIA estimated a "fair" chance that United States'-sponsored Bay of Pigs invasion would succeed. The author of that assessment later said he used "fair" to indicate a 3 to 1 chance of failure.
Following the failure of intelligence around Saddam Hussein's supposed Weapons of Mass Destruction (WMD), which asserted a 100% likelihood that Saddam Hussein was hiding such weapons, the intelligence community finally began to take steps to standardize evidence-based forecasting. In the process they funded Tetlock's research, a few of whose results I bullet-point below.
Tetlock explains the psychology that impairs good forecasting. He identifies two of the broad traits of the best forecasters as (condensed from the book):
- Probabilistic thinking
- A growth mindset
- Numeracy
- Intelligence (averaging around the 80th percentile, so not exceptional intelligence)
- Breaking down problems using the Fermi method
- Frequent updating
- Humility and a willingness (and even eagerness) to admit mistakes
- Aggressively and impartially learning from mistakes
Intriguingly the best forecasters study and learn from their own overreaction and underreaction to new information. They want to improve their calibration to new information. Over and underreaction are not only the primary mistakes of forecasters, but the fundamental processes that create price patterns in markets.If you have reason to think that yesterday's forecast went wrong, there is no glory in sticking to it.
~ Nate Silver
According to a New York Times review, the lessons of "Superforecasting" can be distilled into a few directives for those in the prediction business. Be curious about and work to reduce personal biases. Think in terms of probabilities and recognize that everything is uncertain. Base predictions on data and logic. Unpack a question into its component parts, distinguishing between what is known and unknown, and scrutinizing your assumptions. Keep score of performance and accuracy, and review those past forecasts.
Ironically, Tetlock also points out that some advantages of the superforecasters, like probabilistic thinking, are correlated with lower levels of well-being. A belief in fate leads to greater happiness than a cold-eyed appreciation of reality.
I highly recommend this book. The take-away is better decision making and a superior forecasting process.
Becoming a Market Superforecaster
Most forecasting problems have a defined end-point at which a result is 100% known - for example, "Will the economy shrink next year?" and "Who will win the election?" - But financial markets are self-modifying systems containing numerous positive and negative feedback loops. In market prices there is no endpoint, only a continual process of price discovery. George Soros calls the process of price formation reflexive.Weather forecast for tonight: dark
~ George Carlin
According to Soros, prices influence fundamentals and these newly influenced fundamentals then proceed to change expectations, thus influencing prices. This process is a feature of "Reflexivity," a process that continues in a self-reinforcing pattern. Because prices respond to such feedback loops, market prices veer towards disequilibrium. But sooner or later expectations reverse. This veering from extreme to extreme by means of feedback loops among prices, fundamentals, and expectations explains the familiar pattern of boom and bust cycles.
To improve forecasting in such an environment, investors need to understand the feedback effects among prices (technical analysis), fundamentals (e.g., earnings), and expectations (e.g., sentiment). These feedback effects occur in price underreaction patterns (trends), but also in price overreaction events (price reversion). Like Tetlock's superforecasters, studying the market's over and underreactions to information helps to better forecast its future.The key to making a good forecast is not in limiting yourself to quantitative information.
~Nate Silver
Gold
One of the more psychological assets is Gold, which trades like a currency and is also coveted for its appearance. Gold's post-crisis overreaction (bubble) makes for an interesting case study.All that glitters is not gold;
Often have you heard that told:
Many a man his life hath sold
But my outside to behold:
Gilded tombs do worms enfold.
~ William Shakespeare, The Merchant of Venice
In January 2010 George Soros noted that gold was becoming the “ultimate asset bubble.” He didn't mean that in a negative way. Soros understands that bubbles create excellent opportunities. Soros had accumulated long positions in gold and gold miners at the time of the above quote, but in further testament to his forecasting prowess, he exited his gold positions with a profit before the gold bubble burst.
Commodities are a zero-sum game of sorts - some benefit from price appreciation (the longs and producers), while others suffer (shorts and consumers). In our text analytics work, we find that net references to the direction of the commodity's price direction (climbing or falling) in the media is more often than not a useful indicator of its future direction. The priceDirection TRMI was derived by quantifying the total references to the price of an asset increasing net of references to the price decreasing. In the sense that it captures the chatter about prices, and such chatter predicts future price direction, it is a marker of underreaction.
A plot of the Gold price since 2012, alongside a PriceDirection MACD, is visible below.
Beyond gold, the predictive value of the priceDirection TRMI for Crude Oil was noted in 2012, and it has remained impressively predictive since it was first identified, including forecasting the trends and turns in the crude oil price with excellent timing over the past several years, as updated in last month's newsletter.
There is a prominent slide in the price of Gold in the above chart during Spring 2013. The slide began in November 2012, and by early 2013 that the decline gained speed. Several events supported an emerging consensus that gold was over-valued. Goldman Sachs formally reversed a bullish forecast for gold in December 2012. Then a rumor spread that Cyprus was dumping its 14 tons of gold reserves onto the market. Paul Krugman wrote an Op-Ed in the New York Times (“Lust for Gold”) summarizing the evidence that there had been unsustainable investment in gold. Krugman cited a 2011 Gallup poll in which one-third of Americans called gold the best long-term investment (gold peaked at over $1800/oz in 2011). The gold consensus rapidly turned bearish, and prices followed the consensus lower. Gold declined more than 25% from peak to trough over that 6 month period and over one-third from its 2011 peak to today.
Despite a recent uptick in sentiment and enthusiasm around gold, the price is not responding significantly, and thus the slump looks likely to persist over the next year or so (a 60% probability, to be specific).
Housekeeping and Closing
To best forecast the probability of markets rising or falling, it helps to analyze the patterns of prices in response to information as well as the feedback loops between information, prices, and investor expectations.Part of my advantage is that my strength is economic forecasting, but that only works in free markets, when markets are smarter than people. That's how I started. I watched the stock market, how equities reacted to change in levels of economic activity, and I could understand how price signals worked and how to forecast them.
~ Stanley Druckenmiller
Our next newsletter will feature several remarkable studies on the role of attention in driving over and underreaction when new information hits markets.
We love to chat with our readers about their experience with psychology in the markets. Please send us feedback on what you'd like to hear more about in this area.
Please contact us if you'd like to see into the mind of the market using our Thomson Reuters MarketPsych Indices to monitor real-time market psychology and macroeconomic trends for 30 currencies, 50 commodities, 130 countries, 50 equity sectors and indexes, and 8,000 global equities extracted in real-time from millions of social and news media articles daily.
Always Learning,
The MarketPsych Team