Why Forecasts Dont Work in Finance

By Joanna Higgins

September 16th, 2008

Finance is an area thats dominated by rare events. The tools we have in

quantitative finance do not work in what I call the Black Swan domain, said author and
former options trader Nassim Nicholas Taleb earlier this year.

Events in the financial markets have proven Taleb right at Lehman Brothers and
Merrill Lynch, and from the looks of things, insurance giant AIG.

The problem is that predictive tools in the bankers and insurers arsenal are
unreliable because they calculate the probability of future occurrences by referring
to past events.

Probability and forecasting can let you down when you most need it, observes Rafael
Ramirez, fellow in strategic management at Said Business School in Oxford and co-
editor of Business Planning in Turbulent Times. People believe they can predict the
future by calculating probabilities on whats gone before. But this assumes events fit a
pattern. When turbulence sets in, as now, pattern is irrelevant.

Ramirez is an advocate of scenario planning, which considers a number of alternative

future outcomes and is based solely on now and tomorrow. Scenario planning was most
famously practiced at oil giant Shell, where Arie de Geus and his colleagues applied what
were originally military planning techniques to the business.

Nobodys saying it wouldve saved Lehman Brothers, but its a good fit for todays
ambiguous business environment. Ramirez explains:

Using plausibility, not probability

Scenario planning is what might be we make the implausible plausible. It asks:
under what context could an activity become unviable? Good scenario work is only just
plausible if you start by offering decision-makers probability, they will assume theres
a solution and wont explore possible unexpected outcomes.

Outside perspectives

At Lehman, it was observed that a boss less intimately wedded to his business Mr
Fuld was the longest-serving chief executive on Wall Street may have seent he
problems earlier and acted faster.

Fuld was so embedded in the Lehman world view, he couldnt see the alternative context
in which the set of assets hed built could become a set of liabilities.

This applies to other businesses your business strength may once have been your
corporate culture, but it couldve become a liability at some point. You need people
outside of the business to point this sort of thing out.

Collaboration

The notion that markets will look after themselves doesnt hold up anymore taxpayers,
regulators and businesses will need to work together.

There needs to be a minimum of collaboration among businesses within an industry.

Look at football: if it were really just competition, Manchester United and Chelsea
players would just shoot each other. They collaborate to establish a framework of rules
and standards that make it possible to compete.

Scenarios help to re-establish the collaboration that makes competition viable. The (late)
collaboration among bankers, regulators and central banks attests to this. Scenarios might
have helped this to happen earlier.

Conversation enabler

Scenarios help to raise assumptions on which decision-making is based, then question

them what if?, and then?, what next. Keith Grint talks about how John F
Kennedy did this to the military during the Cuban missile crisis. Its an enabler of
strategic conversations.

When you create scenarios with others, they are not a final product, but invite further
insights.

Succession planning
One bank recently wanted its next generation of leaders to present scenarios to current
ones. This way you have an inter-generational conversation so the younger leaders own
the scenarios theyll inherit, and they have more skin in the game.

(Image: Dullhunk, CC2.0 )

The Black Swan: The Impact of the

Highly Improbable by Nassim Nicholas Taleb
Dean Swift

Taleb has one good idea, a great idea even, and an infinite number of ways of talking
about it. It is essentially the same idea as his last book, Fooled by Randomness: namely
that life does not behave with regularity. Those who think it does, he says, will always be
tripped up by the unexpected. Black Swan extends that idea, beyond the financial markets
he concentrated on in Randomness, to just about all walks of life. He is a magpie for
anecdote and stray pieces of supporting evidence wherever he can find them. He calls all
this 'skeptical empiricism'.
The qualification is that his big idea is not original, though his numerous examples do
help bring home its ubiquity. More problematically, he overstates its usefulness. For when
it comes to calling your next move, the unpredictable and the unexpected are, by
definition, not things we can anticipate. And though he is right that in the long run there
will undoubtedly by high impact improbably events, it is also true, as Keynes said, that in
the long run we are all dead: organizing your life on the principle that something radical
might come along doesn't solve the everyday problem of what to next.
In short, he exaggerates his own insight and the authority it gives him. That's a wicked
irony, for the chief target of his ire is those with an exaggerated sense of insight and
control over their lives.
Oh, and the tone... Taleb wants to be seen as a radical iconoclast. Every sentence drips
righteousness and often irritation. He is the strutting, impatient sage, the rest of us
blinkered morons. Apparently he doesn't like his editors trying to change this. A word of
advice to the author: if you want your advice heeded, don't shout and sneer at your
audience. For this reason, an interesting thesis, but in the end a wearisome read.

2007 Random House, 396 pages (of which 305 pages for main body of book)

Andrew Barrett (UK)

The Black Swan is a very unusual book. A couple of days after finishing it I still feel like
I'm struggling to integrate its message with life. It reminds me a little of Richard
Dawkins' Selfish Gene in that respect: its central thesis, which appears to be unassailably
argued, indicates that the standard view of the world is wrong. In Dawkins' case, the
primacy of the individual (as opposed to the gene) and in Taleb's case, the view that the
world is essentially driven by normal, day-to-day events.

The subtitle of Taleb's book tells you what it is about: The Impact of the Highly
Improbable. According to Taleb, high-impact rare events ('Black Swans') are not anything
like as rare as we think they are and their effect is so disproportionately large that they
effectively drive events in the world.

This may not sound all that provocative, but Taleb's argument is that virtually
everybody's view of the world as essentially linear and step-by-step is just an illusion that
protects us from understanding that our progress through life is much more random and
fragile than we think.

Taleb's outstanding first book, Fooled by Randomness, is about the much greater role of
luck in life than is commonly understood. The Black Swan develops this thesis further
and shows that rare and unexpected events (what you might think of as a subdivision of
luck) drive much of the results in the world.

Since reading Richard Koch's The 80/20 Principle eight or so years ago - after which I
began to look at the world through the lens of unequal cause and effect - I have been
coming gradually around to Taleb's views. Even so, The Black Swan is difficult to
assimilate - and it must seem extremely odd (and itself most improbable) to those who
have not had some preparation.

One can see this difficulty in the absolute refusal of modern academic finance to give up
theories of how the world works (the bell curve or pattern of 'normal' distribution) that
allow them to use complicated mathematics and which are just plain wrong. Taleb
thought the Long Term Capital Management debacle in 1998 - in which various Nobel-
winning economists proved their (Nobel-winning) ideas did not apply in the real world -
would be the end of these dangerously wrong beliefs.

However, it was Taleb who was wrong about that: a whole gang of academics who have
invested a good chunk of their lives in an idea that turns out to be worthless (actually
significantly negative) won't give it up easily (and perhaps not until they are all dead).
The sub-prime mess, with accompanying cries of surprise and of '25-standard deviation
events' from various hog-greedy financiers and hedge fund managers, shows the
continued prevalence of these appalling ideas.

After the success of Fooled by Randomness, it would appear that Taleb had more
freedom to write (and be published) however he wanted. It makes The Black Swan more
idiosyncratic and aggressive than Fooled by Randomness. I imagine this will act as a
polariser and some people who would otherwise appreciate the content may not like the
delivery. Personally, though, I loved it.

As someone who has tried working in various jobs in the City of London (the UK
equivalent of Wall Street) I feel in some ways that Taleb is a kindred spirit: I can't stand
arrogant, ignorant 'empty suits' either. I thought his "Get Another Job" section (p. 163)
was perfect:

"There are those people who produce forecasts uncritically. When asked why they
forecast, they answer, "Well, that's what we're paid to do here."
My suggestion: get another job.
This suggestion is not too demanding: unless you are a slave, I assume you have some
amount of control over your job selection. Otherwise this becomes a problem of ethics,
and a grave one at that. People who are trapped in their jobs who forecast simply because
"that's my job," knowing pretty well that their forecast is ineffectual, are not what I would
call ethical. What they do is no different from repeating lies simply because "it's my
job.""

Taleb's very severe and aggressive criticism of risk measurement techniques in modern
finance could be interpreted as an intemperate rant. I don't subscribe to this view and
suspect Taleb chose this approach deliberately in order to make it clear that the prevalent
financial risk management techniques and his ideas cannot in any way coexist: they are
absolutely and totally mutually exclusive. (Taleb mentions that after finding it impossible
to refute his ideas some people then try to combine them with their old ways of
operating.)

I also liked the way Taleb approached and structured his book: he uses stories to get ideas
across (as with Nero Tulip in Fooled by Randomness) and has separated his book into
sections that allow one to understand his ideas with or without the scientific underlay (I
think this is a great idea).

Some people (whether willfully or not) confused the central theme of Fooled by
Randomness, that much of life is driven by luck, with the superficially similar but totally
different 'all of life is driven by luck'. In a similar way, I believe some people think that
Taleb's message in The Black Swan is unremittingly negative: that we are all permanently
exposed to large unexpected events that can wreck all our plans in an instant, and which
we can do nothing about.

Taleb's point is rather that most specific forecasting is pointless, as large, rare and
unexpected events (which by definition could not have been included in the forecast) will
render the forecast useless. However, as Black Swans can be both negative and positive,
we can try to structure our lives in order to minimize the effect of the negative Black
Swans and maximize the impact of the positive ones.

I think this is excellent advice on how to live one's life and seems to be equivalent, for
example, to the focus on downside protection (rather than upside potential) that has led to
the success of the 'value' approach to investing. Highly recommended.
James Taylor "Enterprise Decision Management ... (Palo Alto, CA USA)

NNT (as he calls himself) has some fascinating points and some interesting turns of
phrase, though he does rather go on and on and on.... Leaving aside the long-winded
somewhat self-absorbed writing style, NNT makes some interesting points that he
illustrates well. He discusses the problems with the fact that humans seek validation for
what they think (rather than challenging themselves) and why we cannot predict well in
many circumstances. He spends a lot of time discussing the problems inherent with the
use of a bell curve to predict things where the impact of extreme events really matter.
Finally he spends (too little) time on what to do about all this. I took a few key points
away from the book:
- It is better, perhaps, to try and be generally right rather than precisely wrong
- Beware of looking for more rules than really exist
- Watch out for a tendency to prepare for "last war"
- Rare and consequential events can be much more important than the "normal" stuff in
the bell curve
- Some things (that he calls "mediocristan") are such that the most typical is average and
single instances don't impact the total much
- Others (he calls these "extremistan") are more winner-takes-all kinds of environments
where extreme events matter most
- He makes the point that a thousand days cannot prove you right but one can prove you
wrong

There's more but it's a very long book and I am not going to attempt to summarize the
nuggets spread throughout it here. Be prepared for a slog if you want to get everything
you can out of the book - it goes on and on. 3 stars to average out 1 star for length and
incoherence in places and 5 stars for some great content.

Death of VaR Evoked as Risk-Taking Vim Meets Taleb's

Black Swan
By Christine Harper
Jan. 28 (Bloomberg) -- The risk-taking model that emboldened Wall Street to trade with
impunity is broken and everyone from Merrill Lynch & Co. Chief Executive Officer John
Thain to Morgan Stanley Chief Financial Officer Colm Kelleher is coming to the
realization that no algorithm or triple-A rating can substitute for old-fashioned due
diligence.

Value at risk, the measure banks use to calculate the maximum their trades can lose each
day, failed to detect the scope of the U.S. subprime mortgage market's collapse as it
triggered more than $130 billion of losses since June for the biggest securities firms led
by Citigroup Inc., Merrill, Morgan Stanley and UBS AG.

The past six months have exposed the flaws of a financial measure based on historical
prices that securities firms use idiosyncratically and that doesn't anticipate every potential
disaster, such as the mistaken credit ratings on defaulted subprime debt.

``Finance is an area that's dominated by rare events,'' said Nassim Taleb, a research
professor at London Business School and former options trader. ``The tools we have in
quantitative finance do not work in what I call the `Black Swan' domain.''

Taleb's book ``The Black Swan,'' published last year by Random House, describes how
people underestimate the impact of infrequent occurrences. Just as it was assumed that all
swans were white until the first black species was spotted in Australia during the 17th
century, historical analysis is an inadequate way to judge risk, he said.
Merrill vs. Goldman
Executives at Merrill, Morgan Stanley and UBS took steps in the past six weeks to
overhaul their risk-management groups after internal models failed to foresee the first
annual decline in house prices since the Great Depression that eroded five years of
trading gains.

Goldman Sachs Group Inc., the firm with the highest nominal VaR, was the sole
investment bank to report record earnings in the fourth quarter, while New York-based
Merrill, which had the second-lowest nominal VaR of the five biggest U.S. securities
firms, posted a $9.8 billion loss for the last three months of 2007, the biggest in its 94-
year history.

Thain, who replaced the ousted Stan O'Neal last month at Merrill, said Jan. 17 that the
largest U.S. brokerage should stop making trades that have the potential to wipe out
profits. He revamped the unit overseeing trading positions and hired former Goldman
executive Noel Donohoe as co-chief risk officer.

UBS CEO Marcel Rohner told employees two weeks ago that Europe's biggest bank will
scale back risk taking after reporting a $15 billion writedown last year for subprime-
infected investments. As part of the plan, Zurich-based UBS shut a U.S. fixed-income
trading unit.

`Necessary Changes'
At New York-based Morgan Stanley, which disclosed a $3.56 billion fourth-quarter loss
after writing down mortgage-related and other securities by $9.4 billion, the risk
department will now report directly to Kelleher instead of to the trading heads. The firm
said it plans to hire more risk managers.

``This loss was a result of an error in judgment that occurred on one desk in our fixed-
income area and also a failure to manage that risk appropriately,'' Morgan Stanley Chief
Executive Officer John Mack said on Dec. 19. ``We're moving aggressively to make
necessary changes.''

Stronger management might help protect firms against rogue employees. Societe
Generale SA, France's second-largest bank by market value, said last week that
unauthorized trades caused a $7.2 billion loss, the biggest in banking history. The trading
wiped out about two years of pretax profit at the Paris-based company's investment-
banking unit.

Rogue Trader
Societe Generale said the trader was Jerome Kerviel, 31, who joined the bank's trading
division in 2006 after working six years in the company's back office. He had ``intimate
and perverse'' knowledge of the bank's controls, which enabled him to avoid detection,
Co-Chief Executive Officer Philippe Citerne told reporters on Jan. 24. Citerne didn't
identify the trader.

``That's a whole other side of risk management,'' said Benjamin Wallace, who helps
oversee $850 million at Grimes & Co. in Westborough, Massachusetts, which owns
shares of Morgan Stanley and Merrill. ``That's not the modeling, that's making sure that
people are overseen properly.''

Hiring risk managers and giving them more power won't alter the mistake that led to last
year's slump and that was Wall Street's dependence on statistics to quantify risks, Taleb
said.

``We have had dismal failures in quantitative finance in measuring these risks, yet people
hire quants and hire risk managers simply to back up their desire to take these risks,'' he
said. ``There are some probabilities that you cannot compute.''

Bigger Bets
Banks and securities firms increased the size of their trades during the past decade on
interest rates, stocks, commodities and credit. Trading revenue for the five largest
securities firms -- Goldman, Morgan Stanley, Merrill, Lehman Brothers Holdings Inc.
and Bear Stearns Cos. -- climbed to a combined $71.1 billion by 2006 from $29.1 billion
in 2002. The higher profits added to the firms' capital, enabling even bigger trading bets.

``You can scale your value at risk relative to your book value,'' said Wallace. ``It was a
self-reinforcing part of the cycle.''

Goldman's average daily VaR more than tripled to $151 million in the fourth quarter from
$46 million five years earlier, according to company reports. Goldman's VaR was almost
twice as high as Merrill's in the third quarter.

Merrill said third-quarter daily average VaR was $76 million, compared with Goldman's
$139 million, Morgan Stanley's $87 million, Lehman's $96 million and Bear Stearns's
$32 million.

Different Methodologies
All the New York-based firms base their calculations at a confidence level of 95 percent,
meaning they don't expect one- day drops to exceed the reported amount more than 5
percent of the time.

The amounts differ in part because every firm uses their own methodology and data. For
instance, Lehman uses four years of historical data to calculate VaR, with a higher
weighting given to more recent time periods, while Morgan Stanley provides VaR
calculations using both four years and one year of market data.
``If you compare what peoples' values at risk are versus what their losses were in the
third quarter or fourth quarter, the numbers are astounding,'' said David Einhorn,
president and co-founder of hedge fund Greenlight Capital LLC in New York. ``There are
a lot of things that probably the value-at-risk model said would have trivial losses 95
percent of the time or 99 percent of the time but are now having a huge loss.''

Mortgage Defaults

Merrill's highest one-day value at risk in the third quarter was $92 million, indicating that
the firm's maximum expected cost during the 63-trading day period would be $5.8
billion. In fact, the firm wrote down $8.4 billion from the value of collateralized debt
obligations, subprime mortgages and leveraged finance commitments, 45 percent more
than the worst- case scenario.

All of the risk-measurement tools failed to prepare Merrill for the unforeseen declines on
triple-A rated securities backed by subprime mortgages, according to the company's
third-quarter filing with the U.S. Securities and Exchange Commission. The firm's
writedowns related to the highest-rated portions of CDOs backed by pools of home loans,
which plunged in value as defaults on the underlying mortgages soared.

``VaR, stress tests and other risk measures significantly underestimated the magnitude of
actual loss from the unprecedented credit market environment,'' Merrill's filing said. ``In
the past, these AAA ABS CDO securities had never experienced a significant loss in
value.''

Meriwether's Fund

Securities firms developed statistical models during the early 1990s to better quantify
risks as the trading of bonds, stocks, currencies and derivatives increased. J.P. Morgan &
Co., now part of JPMorgan Chase & Co., helped popularize the use of value at risk as the
primary measurement tool in 1994 when it published its so-called RiskMetrics system.

Four years later, two events helped demonstrate the drawbacks in using statistical
analysis based on historical market movements to measure risk. Russia's bond default
sent fixed-income markets into a tailspin and Long-Term Capital Management LP, the
Greenwich, Connecticut-based hedge fund run by former Salomon Brothers trader John
W. Meriwether, had to be bailed out after $4 billion of trading declines.

`Market Stress'

Russia's default risk was underestimated because value-at- risk computations used by
investment banks depended on market events of the preceding two to three years, when
nothing similar had occurred, according to Wilson Ervin, who's now chief risk officer at
Zurich-based Credit Suisse Group, Switzerland's second-biggest bank after UBS.
Long-Term Capital Management, which amplified its risk by relying on borrowed money
for most of its trading bets, blew up in part because it didn't anticipate that investor panic
after the Russian default would cut the value of any risky debt, whether it was issued by a
country, sold by a company, or backed by mortgages.

The riskiest Russian and Brazilian bonds owned by the fund plunged far more than the
safer Russian and Brazilian bonds that it had bet against as a hedge, according to ``When
Genius Failed,'' the book written by Roger Lowenstein.

``In a market stress event, some individual sectors that previously appeared unrelated do
move together, and as a result, the organization could take losses on both of them or even
on positions that were previously deemed to be a hedge,'' said Ed Hida, the partner who
runs the risk strategy and analytics services group at Deloitte & Touche LLP in New
York.

Worst-Case Scenario

The other risk tool commonly used by securities firms, known as stress testing or
scenario analysis, also failed to prepare the industry for the plummeting value of AAA-
rated securities that had previously been deemed the most creditworthy, he said.

``Stress tests are only as good or as predictive as the scenarios used and in many cases the
scenarios that played out were much more severe than people anticipated,'' Hida said.
``One lesson learned is that these stress tests should be broader, should consider more
scenarios.''

Kelleher, who became Morgan Stanley's CFO in October, explained the flaw in the firm's
stress testing in a Dec. 19 interview, the day the company reported its first unprofitable
quarter.

``Our assumptions included what at the time was deemed to be a worst-case scenario,'' he
said. ``History has proven that the worst-case scenario was not the worst case.''

Credit Suisse

At Credit Suisse, one of the firms that have so far skirted the worst subprime declines,
Ervin said value at risk played no role in helping him navigate the market turmoil.

``Once you go into a crisis like this, I think risk is much more about sitting down with
traders, and talking about very specific issues and scenarios,'' he said. ``VaR we know
will kind of lag going into a crisis so we don't really watch that as a crisis indicator.''

Still, Ervin said VaR provides a service if used every day because it can pick up
fluctuations in the risk that the firm is taking in some distant region or an arcane product
that might not otherwise be noticed.
Investment banks will continue to take unsafe risks as long as traders are rewarded for
making profits, leaving shareholders, bondholders and sometimes taxpayers to shoulder
the consequences, Taleb said.

Wall Street traders ``make an annual bonus and get an annual review based on risks that
don't show up on an annual basis,'' Taleb said. ``You have all the incentive in the world to
take these risks.''

To contact the reporter on this story: Christine Harper in New York at

charper@bloomberg.net.

Last Updated: January 27, 2008 19:12 EST

Business Planning in Turbulent Times:

New Methods for Applying Scenarios (Science in Society Series):
New Methods for Applying Scenarios (Science in Society Series)
(Hardcover)

by Rafael Ramirez (Author, Editor), John W. Selsky (Editor), Kees van der Heijden
(Editor)

Product Description
The world is an increasingly turbulent and complex environment, awash with tipping
points and knock-on effects ranging from the impact of warfare in the Middle East on
energy futures, investment and global currencies to the vast and unpredictable impacts of
climate change. This book is for business and organizational leaders who feel the ever-
increasing turbulence of the environment and are interested in thinking through how to
deal with related complexity and uncertainty. The authors explain in clear language how
future orientation, and specifically modern scenario techniques, help to address
increasing risk and lead to more confident and robust decisions. They draw on examples
from a wide variety of settings and circumstances including the large corporations, inter-
governmental organizations like the World Bank, small firms, municipalities and other
communities. Readers will be inspired to try out scenario approaches themselves to
address the turbulence that affects them and others with whom they work, live and do
business. A key feature of the book is the exchange of insights across the academic-
practitioner divide. What has previously remained jargon only accessible to the highest
level of corporate and government futures planners here becomes comprehensible to a
wider business and practitioner community.

Synopsis
The world is an increasingly turbulent and complex environment, awash with tipping
points and knock-on effects ranging from the impact of warfare in the Middle East on
energy futures, investment and global currencies to the vast and unpredictable impacts of
climate change. This book is for business and organizational leaders who feel the ever-
increasing turbulence of the environment and are interested in thinking through how to
deal with related complexity and uncertainty. The authors explain in clear language how
future orientation, and specifically modern scenario techniques, help to address
increasing risk and lead to more confident and robust decisions. They draw on examples
from a wide variety of settings and circumstances including the large corporations, inter-
governmental organizations like the World Bank, small firms, municipalities and other
communities. Readers will be inspired to try out scenario approaches themselves to
address the turbulence that affects them and others with whom they work, live and do
business. A key feature of the book is the exchange of insights across the academic -
practitioner divide.

What has previously remained jargon only accessible to the highest level of corporate and
government futures planners here becomes comprehensible to a wider business and
practitioner community. This book unlocks the high-powered secrets of scenario planning
techniques - until now the reserve of large corporate players, governments and militaries -
for a wide audience of business and organizational leaders. It presents powerful tools for
all businesses or organizations in today's increasingly risky, complex and turbulent
environments. It is authored by the world's leading scenarios experts, including Kees van
der Heijden, author of the bestselling business book "Scenarios: The Art of Strategic
Conversation".