Taking the risk out of systemic risk measurement

2014 American Economic Association Meetings Philadelphia, PA Levent Guntay, FDIC leguntay@fdic.gov Paul Kupiec, American Enterprise Institute paul.kupiec@aei.org

The Search for Systemic Risk

The search for systemic risk measures is all about big business
It focuses on big complex financial businesses It is a big business opportunity for financial economists

But has it really identified systemic risk?

There are Big Risks in the continued use of some currently popular systemic risk measures
Should government require a mandatory warning label?

We Focus on Two Measures CoVaR and MES (aka SES & SRISK)
CoVaR
Conditional Value at Risk
The value at risk of a conditional stock return distribution Adrian and Brunnermeier, (2011) CoVaR, FRB of New York. Staff Report No. 348.

MES

Marginal Expected Shortfall

The expected shortfall of a conditional stock return distribution Acharya, Engle, and Richardson, (2012). Capital Shortfall: A New Approach to Ranking and Regulating Systemic Risks, The American Economic Review 102, 59-64. Acharya, Pedersen, Philippon, and Richardson, (2010). Measuring Systemic Risk, Technical report, Department of Finance, NYU Stern School of Business.

Both CoVar and MES Measure Tail Dependence

CoVaR and MES measure an institutions systemic risk
Large CoVaR or MES estimates are interpreted as evidence that an institution has high systemic risk potential

Intuition: Systemic risk will generate tail dependence in financial institutions stock returns
Catastrophic losses in an important financial institution will induce
Large losses in other financial institutions
Defaults, counterparty losses, interconnectedness, contagion

Large losses in nonfinancial firms stock returns

Financial sector meltdown will strangle credit supply to the real economy

Warning!
CoVaR and MES Confound systemic and systematic risk
Firms with large systematic risk components have large CoVaRs and MESs

They diagnose systemic risk without a proper hypothesis test

Literature argues that firms that failed or needed govt assistance during the financial crisis had large CoVaRs or MESs prior to the crisis
Concludes large CoVaR or MES for a large financial institution==systemic risk

But the literature has no formal hypothesis tests!

CoVaR and MES measures can be calculated for all firms

Real-side firms can have larger CoVaRs and MESs

CoVaR, MES and Systematic Risk

Cross section of CRSP stock returns 2006-2007 Run regression on MES on MM Beta Run regression of CoVaR on Market correlation Large Beta, Large market correlation = Large (negative) MES, CoVaR

Contribution to the systemic risk literature

We introduce a proper null hypothesis
Stock returns are Gaussian

This allows us to:

Separate systemic risk from systematic risk Formulate a classical hypothesis tests for presence of systemic risk

Stock returns and tail dependence

Gaussian returns --independent in the tails of the distribution & symmetric
Very large/small realization in one dimension does not increase the probability of a very large/small realization in the other dimension If returns are Gaussian, there is no systemic risk

Systemic risk hypothesis-> stock returns have left-tail dependence

When financial firms suffer large losses, there is a higher probability that other firms (financial and real) will suffer large losses With systemic risk, returns are not Gaussian

How large must tail dependence estimates be before we can reject the null hypothesis of no tail dependence?
Need a proper statistical test

When returns are Gaussian

CoVaR = ( MES | < ( , 95%) =
1

.01,

= 2.32635
1.645 1.645

= 2.32635

= 2.062839

Our test strategy uses 2 estimators

Gaussian (parametric) estimators
CoVaR and MES are calculated from sample estimates of the mean, std dev & covariance These estimates do not allow tail dependence Unbiased and efficient if Gaussian null hypothesis is true Biased if alternative hypothesis is true

Nonparameteric estimators

CoVaR is estimated using quantile regression focusing on the 1% quantile

MES is estimated as the average stock return on days when the market return is in its 5 percent left-hand tail If null is true, nonparametric estimators are unbiased but not efficient If alternative is true, nonparametric estimators are still unbiased The nonparametric estimators can produce much larger negative CoVaR or MES estimates if there is tail dependence in returns

The 1% quantile of the CRSP equal-weight market portfolio conditional on stock js return equal to its 1 percent quantile

Sampling Distribution of CoVaR Estimators

Parametric High correlation example
Correlation .95 Mean CoVar PCoVar Correlation -.0089 -.0088 .381 Std. Dev. .0008 .0003 Sigma i=.004 Q05 -.0103 -.009 Rank Correlation Q95 -.0007 -.008

Nonparametric

.375

CoVaR Sampling Distribution 2

Parametric Low Correlation Example
Correlation .05 Mean CoVar PCoVar Correlation -.0005 -.0005 .288 Std. Dev. .0016 .0004 Sigma i=.004 Q05 -.003 -.001 Rank Correlation Q95 .002 .0002

.2567

Nonparametric

Test Statistics
Our tests evaluate the difference between two estimators Nonparametric estimate-Parametric estimate
CoVaR MES
Quantile regression CoVaR estimate-Gaussian CoVaR estimate Selected sample average MES estimate-Gaussian estimate

Both estimators are unbiased under the null If Null is true, parametric is most efficient estimator

The differencing controls for systematic risk We scale these differences to remove idiosyncratic risk dependence
CoVaR difference is scaled by Gaussian CoVaR estimate MES is scaled by estimate of stock idiosyncratic standard deviation

Correlation remains as a nuisance parameter We calculate critical values for these test statistics using Monte Carlo simulations

Test Statistic Critical values

Sample size =500 obs .about 2 years of daily data 25,000 Monte Carlo replications

Apply test to CRSP daily returns: 2006-2007

Table 6: Industry Representation in Sample Financials Depository Institutions Insurance Other Financial Broker Dealers Non-financials Manufacturing Services Transportation, Communication, Utilities Retail Trade Mining Wholesale Trade Construction Public Administration 1,324 626 317 224 144 110 42 13 380 139 101 55

Results
Lots of firm returns reject the null
Many more rejections are nonfinancial than financial

MES and CoVaR often disagree about which firms are potentially systemic
MES test rejects the null much more frequently than CoVaR test

Some summary pictures of results by industry.

Banks (Depository Institutions)

Rejection region

Insurance Industry

Rejection region

Retail Trade

Rejection region

Manufacturing
Rejection region

CoVaR and MES Often Identify Different Firms as Systemic

Top 25 BHCs Systemic Risk Measures 2006-2007 by Market Cap in 2006

Summary & Conclusion

Our contribution is to introduce a null hypothesis into systemic risk modeling
Removes systematic risk contamination in systemic risk measurement Needed to for classical hypothesis tests (much needed in this literature)

Tests must be improvedviolations may not indicate systemic risk

The Null hypothesis is too restrictive Many data generating processes could lead to rejection, even if they dont allow for tail dependence and systemic risk

Can test idea can be extended to systemic risk measures based on CDS-spreads?