Book Review: The Failure of Risk Management by Douglas Hubbard

Douglass Hubbard, in The Failure of Risk Management: Why It’s Broken and How to Fix It, presents a strongly worded critique of qualitative risk management methodologies. For something that might be perceived as dry as risk management, Hubbard’s disdain for most of those who practice non-mathematical risk analysis keeps this ultimately persuasive book entertaining. While his vitriol sometimes detracts from the rhetorical effectiveness of his argument, one comes to understand his frustration at those who pedal unproven, possibly worse-than-useless risk analysis techniques. This is a field, as we’ve seen recently with both the global financial meltdown and the BP Gulf of Mexico oil spill, where failures have the potential of having effects far beyond individual firms. With all his focus on debunking common qualitative methods, though, Hubbard doesn’t stop there. He ultimately mounts a defence of quantitative modelling (so excoriated by many in the wake of the sub-prime debacle) and makes some very useful, practical suggestions on how to use qualitative models (such as Monte Carlo and Bayesian analysis) effectively.

In order to examine how successful risk management has been, Hubbard divides the approaches to risk management into four categories based on who originally devised them:

* Actuaries
* War Quants
* Economists
* Management Consultants

Hubbard sees the first three as comprising progressively more sophisticated methods of analysis. From the actuarial tables of insurance companies, to Probabilistic Risk Analysis (PRA) devised to predict failures in complex war logistics (and which lead ultimately to Monte Carlo analysis, of which Hubbard is a devotee), to the MPT (Modern Portfolio Theory) of Harry Markowitz and options theory of Black, Scholes and Merton, Hubbard sees the increasingly probabilistic yet thoroughly quantitative views of risk as being all to the good. The problem with these methods, however, is that they are not always intuitive and can be difficult to perform (anyone who has ever tried to do Monte Carlo analysis without the aid of a computer will attest to this).

This difficulty leads to what Hubbard really thinks is the real problem with risk management: management consultants. As a consultant myself, I try not to take this personally, but Hubbard’s analysis is spot-on. The ability to make people believe there is a real analysis happening, yet distilling it into easy, Powerpoint-ready chunks, is the real talent of many consultants. Drawing from his own experience as an MBA consultant with Coopers & Lybrand in 80s, he points how “structured methodologies,” which have all the appearance on being based on proven theories and which are easily graspable by non-technical senior managers, are not only doing no good for their clients but mask serious risk issues. An example of this is the typical matrix chart, such as this:

He argues that there is no evidence that these sorts of analytical tools do anything constructive, beyond giving management the feeling that something is being done about risk. He ironically offers a set of suggestions on how consultants should sell their useless wares:

Sell what feels right. Clients will not be able to differentiate a placebo effect from real value in most risk management methods. The following tricks seem to work to produce the sense of value:

Convert everything to a number, no matter how arbitrary. Numbers sound better to management. If you call it a score, it will sound more like golf, and it will be more fun for them.

As long as you have at least one testimonial from one person, you are free to use the word proven as much as you like.

Use lots of “facilitated workshops” to “build consensus.”

Build a giant matrix to “map” your procedure to other processes and standards. It doesn’t really matter what the map is for. The effort will be noticed.

Optional: Develop a software application for it. If you can carry on some calculation behind the scenes that they don’t quite understand, it will seem much more like magic and, therefore, more legitimate.

Optional: If you go the software route, generate a “spider diagram” or “bubble chart.” It will seem more like serious analysis.

The net result of this has been that the most popular risk management methodologies today are developed in complete isolation from more sophisticated risk management methods known to actuaries, engineers, and financial analysts. Whatever the flaws of some of these quantitative methods, the methods developed by the management consultants are the least supported by any theoretical or empirical analysis.

(You see what I mean about how his presentation lacks even the appearance of impartiality).

Hubbard traces the difficulty with risk management, and the ability of people to imagine that non-quantitative methods are in any way useful, to the lack of consistent definitions of the terms uncertainty, ignorance, unknowability and risk. While this might seem like an esoteric discussion for what aims to be a practical book on how to fix risk, it really does get to the heart of the matter. I have to admit that it’s refreshing to see someone take on the specialist definitions of risk because, as someone who studies risk, I’ve been confounded by the various definitions from finance, project management, general line management, and others. I remember first encountering the finance definition of risk as being equivalent with volatility when dealing with the Capital Asset Pricing Model (CAPM) doing M&A consulting and thinking, “ok, that’s a stipulative definition that I’m not familiar with but it doesn’t seem really to mean the same thing normal people mean when they use the word ‘risk’”. Hubbard eventual argues for a definition that:

includes some probability of a loss
involves only losses, not gains
is not synonymous with volatility (outside finance)

The eventual and ultimate aim of this book is to argue in favour of quantitative and probabilistic methods of describing and analysing risk. The early parts of the book, which debunk qualitative methods and clarify the confused (and confusing) terminology in the field, are necessary to clear the way for him to advocate for methods like Monte Carlo modelling and Bayesian analysis. Along the way, however, I think he makes a very good point about the usefulness of quantitative models and their defensibility with regards to the financial crisis that sprung from the subprime debacle of 2007-2008. Since I was VP of strategy for one of the banks that was most effected by the meltdown and a direct customer of those models, I’ve spent a lot of time thinking about the models that the investment bankers used to predict the viability of certain kinds of loans. I specifically remember sitting in a meeting with one such group of bankers who were explicitly asking us for zero down, no-documentation loans, thinking “that’s crazy…there’s no reason for a no-documentation loan except to lie about something. If you’ve got income, you’ve got documentation”. When I inquired about how such a loan could possibly work, they assured me that they had mathematical models that showed that, if packaged together in a certain way that spread the risk throughout a securitised pool of loans, it ended up being safe as houses. As it were. I was new in my job so I kept quiet and figured there was just something I wasn’t getting.

But when the subprime market imploded and took the world economy with it (after I’d moved on to consulting), a great deal of criticism was pointed the way of the investment bankers and their mathematical models. People like Nassim Nicholas Taleb, (author of The Black Swan and Fooled by Randomness) and Michael Lewis (author of The Big Short and Liar’s Poker) argued that the models themselves were flawed because, among other things, they didn’t/couldn’t take into account uncertainty and randomness. In fact, Taleb argues, the models can’t possibly adequately describe risk and therefor should be scrapped in favour of more intuitive approaches that take into consideration extremely unlikely events. The blind adherence to complex mathematical models with little correlation to the real world, many people argued, is what caused the sub-prime meltdown, as it had caused the crash-and-burn of Long Term Capital Management a decade before.

It turns out that it wasn’t the models themselves that were wrong, Hubbard and others have argued persuasively. If used properly and with the right data, quantitative risk models would have been able to predict that the sub-prime market would fail. The problem was that they were using models with short-term inputs (data from the previous decade, when the real estate market did nothing but rise) and unwarranted assumptions (if there is a correction, it will be minor). As I’ve pointed out in my blog before, the chief mistake of the those investment bankers was assuming that the market would always go up. Without that assumption (which was authenticated by the data of the late 90s and early 00s), the models would have shown that there was a good chance that the sub-prime securities would go very bad, indeed.

Hubbard understands that even the most sophisticated and structurally correct risk models are susceptible to bad inputs, which is why he spends much of the later portions of the book describing how to avoid the errors in judgment and estimation that lead to such inputs. In doing so, he significantly strengthens his case for building these complex model. He even takes on some of the issue raised by behavioural economists, because if people do not act rationally, it is hard to build models suggesting that they do. While they might not act rationally, the act predictably, which is all one really needs to build models.

The conclusion of The Failure of Risk Management: Why It’s Broken and How to Fix It deals with several practical matters that help people build accurate, complete models for risk, including creating cultural and management incentives for accuracy. This is a must-read for risk professionals, especially those of us for whom Hubbard reserves his special disdain. I have already found myself returning to some of the tools I used in the past (Monte Carlo simulations) and exploring new ones (Bayesian analysis). Given the stakes we sometimes deal with, my hope is that other strategy and risk analysts will be similarly convinced to adhere to the proven methods outlined by Hubbard.

The Ethics Of, and Response To, "Strategic Default" (part 3 of 3)

Part 3: How lenders should handle the situation

So, if strategic default is clearly unethical, financially irresponsible and certainly legally questionable, how are the banks to respond? In the days before the sub-prime meltdown, this would have been an easy question. Since banks were assumed to have the superior ethical standing, they could afford to take a hard-line approach to defaulters and quickly use whatever legal and financial means at their disposal. Now, however, the ethical presumption lies with the borrowers, so the lenders have to be very careful about how they respond.

We will need to approach this issue sensitively and creatively. I have to admit that my first reaction upon hearing about this trend was neither sensitive nor particularly creative. Given the ethical in-defensibility and dire consequences of strategic default, it’s an understandable reaction. Lenders are going to have to resist the urge to play hardball with these borrowers, however. For one thing, it probably won’t work. Since debtors prison hasn’t existed for two hundred years, the borrowers already know that the worst that can happen is that their credit will be ruined; they’ve already worked that into the cost benefit analysis.

The first step will be to draw a profile of who the strategic defaulters are. Since no one bank is likely to have enough data to draw solid conclusions from, there will need to be inter-bank or, preferably, industry-wide cooperation. I’m assuming, since I don’t have access to that sort of data, that a statistical picture of the strategic defaulter is possible and that a predictive model could be constructed. Again, this would have to be an industry effort to aggregate enough data to make meaningful predictions.

Once the picture of the likely defaulter is developed, banks would need to develop individual ways of dealing with them. The most effective way will be to proactively approach the borrowers that fit the profile and offer them some kind of pre-emptive workout plan. It will be harder for the borrowers to default if they feel they’ve been treated fairly by their banks. It would be a way of building brand loyalty and it would skew the cost benefit analysis toward keeping their homes. While it will cost the lenders in the short run, given the fall in value in some real estate markets, the cost of foreclosure and disposition of the defaulted properties will likely even out. Beyond that, should the effects of strategic default become systemic, the losses would be far greater. Dealing with this problem is going to require foresight and a long-term view of these assets and the market as a whole.

Once these proactive workout plans are offered, there will be several further practical steps required to adequately address this problem. First, so that strategic defaulters do not get lumped into the category of those who merely borrowed beyond their means or were victims of the economic downturn, additional means testing would have to be added to work-out planning by the banks. This adds a further burden to non-strategic defaulters, but this is one further consequence of the strategic defaulters’ actions (in other words, the banks can’t be blamed for this…blame the people walking away from their homes).

Second, government regulators and legislators should be brought into the conversation. This sort of activity by individuals borders on fraud and theft and while it may not be universally popular, the systemic danger of strategic default becoming widespread and having a serious impact on the functioning of the lending ecosystem transcends short-term popularity.

Finally, a coordinated publicity campaign about the dangers of strategic default to the society at large and its ethical indefensibility should be undertaken. If the presumption of ethical activity could be more neutral (i.e., neither with the banks nor with individuals), the peer pressure that used to exist that kept people from walking away from their homes could be revived. While it’d require careful execution, I think the intuitive reaction of most people to strategic default is negative, so merely pointing out the possible dire consequences of such defaults might be enough to turn public option strongly against it.

It goes without saying that all three responses would have to be cautious, careful, and sensitively planned and executed. Banks and other mortgage lenders have fallen into the same category as lawyers and politicians in terms of public esteem and ethical estimation. Because of the possibility of systemic consequences such as the possible disruption of the entire mortgage ecosystem (which would have ripple effects in real estate, building, tax revenues, and ever other system that depends on a functioning lending system), I don’t think it should be ignored. The challenge will be convincing the public, regulators, and government officials that the concern of the banks isn’t merely with coughing short-term gains out of already struggling homeowners. By offering proactive help to borrowers who could fall into strategic default, lenders could demonstrate that they are working in the best interest of not only their customers but the economic wellbeing of the entire home owning ecosystem.

The Ethics Of, and Response To, "Strategic Default" (part 2)

Part 2: The Ethical and Systemic Consequences of Strategic Default

For this reason, mortgage lenders should be very cautious in their approach to dealing with this trend. Additionally, there is reason to think that there could be systemic effects should this trend become widespread. If strategic default were to go to 2 million, the risk profile of mortgage lending in the US would change so significantly that it could result in much higher interest rates for all borrowers or, in an extreme case, could see lenders exiting the business for lower-risk places for their capital.

To see why I think there could be systemic effects from strategic default, it’s necessary to understand the risk sharing profile of home lending. There are several kinds of risk involved in making a loan, market, default, and financial. Market risk is the possibility that the property being bought by the homeowner will decrease, rather than increase in value (or, perhaps, even just fall short of the cost of capital, in this case the interest rate of the home loan). Default risk is possibility that the borrower might not repay the loan. Finally, financial risk is the possibility that money the bank uses to lend to the homeowner will become more expensive than loan itself (i.e. the bank’s cost of capital exceeds the interest rate on the loan.)

As in most financial transactions, the holder of the risk gets the rewards for that risk. In the case of mortgage lending, if the house goes up in value, the homeowner gets to keep difference (minus taxes, etc.). If the loan gets repaid, the bank (or holder of the loan in the secondary market) gets compensated for the default risk by interest payments. The same goes for financial risk. If the opportunity cost of the loan was lower than that of other investments, it’s a good investment.

The major problem with strategic defaults, from both ethical and financial perspectives, is that they transfer the market risk from the party who gets the benefit if the risk comes good (the homeowner) to a party who gets gets no benefit and in fact sees nothing but downside risk (the lender). Since the lender isn’t expecting the downside market risk when making the loan, that risk isn’t priced into the loan. The lenders would want either to share in the upside risk (by sharing in the profits of a sale if the market went up) or price the market risk into the into the interest rate (by raising it considerably). But since neither of these scenarios was taken into account went the loans were made, I’d argue that transferring risk to a party that isn’t compensated for it is unethical.

Even more concerning than the losses to banks is the possibility of strategic defaults becoming widespread. While I’m not one to prophesy doom for the worlds most resilient financial system, strategic defaults could have a severely destabilizing effect on the mortgage lending market if their prevalence rises. Without structural changes in the way home loans apportion risk and reward, mortgage lending would become a very unattractive business indeed.

This could play out in a number of ways. Risk-averse investors/lenders could leave the market. The process and qualifications for the getting a home loan could be much more stringent, pushing housing prices down. The most likely scenario would be a wholesale rise in interest rates to account for increased default risk. Here’s where the strategic defaulters would be forcing us into a “tragedy of the common” situation, thereby doubling their ethical . The short-term individual gains of the defaulters would cause a structural change in the way interest rates are calculated. This would result in everyone paying higher interest rates for their loans.

In short, from an ethical perspective, the strategic defaulter is a triple loser. First, they break a contract made in good faith. Second, they cause others to become unable to get home loans as banks tighten their lending standards. Third, as the price of strategic defaults gets priced into the interest rates charged on home loans, they cause many people who are ethically untainted pay the price of their default.

This raises the question, of course, if there are ethically defensible reasons for breaking a contract like a home loan. While this may controversial in some cases, it’s pretty clear that there are situations when a borrower shouldn’t be held ethically (or financially, for that matter) liable for walking away from a loan. If the loan was made under fradulent pretenses (by the lender, that is), for example, a borrower shouldn’t be held to the contract. Similalry, if undue or inappropriate pressure or misinformation was used to get a borrower to take out the loan, I think we’d say they should be released from responsibility. Also, the ethical approbrium we would level at someone who has lost a job or has seen their ability to pay a loan decreased through either macro- or micro-economic circumstances would be diminished. Thus, there are clearly cases when it’s not ethically questionable to default on a home loan and banks should be clear about dilineating the different classes of borrowers when dealing with the default.

That said, under no traditional ethical framework would strategic default be approved of. Clearly, all foundational ethical systems have “no stealing” and “no lying” dicta which preclude strategic default. Virtue ethics, in which one asks if this is something a virtuous person would do, is (as always) a little less definite on this question, but it’s hard to imagine a person walking away from a contract for purely economic gain being considered a paragon of virtue. A deontological approach, especisally that of Kant’s Categorical Imperitive, would clearly make strategic default unethical. In Kant’s approach, we try to imagine a world in which everyone acted in the way we are examining. If it becomes impossible, than Kant would say it’s unethical. Obviously, if everyone defaulted on a loan that lost value, no one would make loans any more, which would make taking out loans impossible. Finally, under a pragmatic/utilitarian ethical framework, which looks at the action in terms of how well it works out for the everyone, strategic default would be considered unethical because, due to the “tragedy of the commons” effect, far more finanical pain is caused by the defaulters than is gained by them.

Tomorrow, part 3: Playing Heavy and/or Creative Solutions: What Should the Banks Do?

The Ethics Of, and Response To, "Strategic Default" (part I)

My reflections on this topic ended up being far too long for a single blog post, so I’ve divided it into three for easier reading. I’ll post them over the next three days).

Part I: Strategic Default: What is it, and why would anyone consider doing it?

Banks, once the paradigm case of fiduciary responsibility, have taken quite a reputational hit in the past half-decade. Where once the banker was the very stereotype of responsible, conservative financial management, whom you’d trust with your money more than you’d trust yourself, many people now see banks as just one more institution interested in nothing more than taking your money from you. While this is a broad generalization, it’s not without some basis in fact. Mortgage lenders and mortgage brokers, particularly, played a substantial role in the current financial crisis. However, an emerging trend called “strategic default,” where homeowners who are capable of repaying loans walk away from their homes because of a drop in investment value, takes unfair advantage of this perception and could possibly endanger the entire home loan ecosystem. Because of the recent reputation of dirty-dealing in lending, though, banks need to very careful in how they analyze and respond to this trend.

Strategic default, as it has come to be known, is when a homeowner who has the means to pay his or her mortgage stops paying their mortgage and allows the bank to foreclose. For example, let’s say a family took out a $400,000 loan in 2005 to buy a house at the peak of the market. They were able to pay the loan at the time and it is still within their means to do so. But let’s further assume that the value of the house, should they sell it today, would be $250,000 (a drop that would be entirely likely in some parts of California). The “strategic” defaulter, seeing that they have $-150,000 in equity, decides that letting the bank have the house and taking the 7 to 10 year hit on his or her credit is worth the $150,000. They know it would probably be 7 to 10 years before they recover the original value of the house, to say nothing of the thousands payed in interest.

Looked at from a purely financial point of view, it’s easy to see their point. Let’s look at the math. In May 2005, the average interest rate for a 30 year fixed rate mortgage was 5.75. Let’s use the following assumptions:

House price: $420,000

Down payment (5%): $20,000

PMI: .5%

Taxes: 1.25%

Current value: $250,000

Rate of return for the next 10 years: 4%

A homeowner would be likely to analyse his or her cost benefit analysis as follows:

The value of the house at the end of the period: $370,000

Total payments: $418,000

Investment value $-48,000

It should be pointed out that this isn’t a very sophisticated way of doing a cost benefit analysis, but I think it is pretty close to the intuitive way people think about their housing investment. A good CBA would consider the pre-2010 payments sunk costs, take into consideration the value of housing for that period, and would include transaction and opportunity costs, etc. But still, I suspect it’s this sort of analysis that leads people to walk away from their homes. They think that for the $418,000 of payments they will have made, a $48,000 loss is a pretty bad return. And who could blame them for thinking that? But a mortgage is not just a financial, but a legal and ethical transaction, as well.

While there has likely always been a small percentage of homeowners who default on their loans for investment reasons, this number has remained small for few reasons. First, since the early 90s, the real estate market has almost always been a good investment. Over that period, there was never a period where the overall market dropped ( see graph for illustration of this phenomenon). Second, homeownership has traditionally held an exalted place in the American psyche and there has always been a good deal of social pressure to own a home among the middle and upper classes. Finally, in a another manifestation of social pressure, anyone who walked away from a home for purely financial reasons would be ostracized as a thief or, at the very least, a deadbeat.

The 20-40% market drop, in some places more, removed the investment incentive. While this has happened before, however, and there was not a wholesale abandonment of homes. The primary difference is that the fallen reputation of banks has provided a sort of excuse for people who would otherwise have been ashamed to walk away from their homes. The assumption of superior ethical standing has moved from the lenders to the borrowers. Now a person who is foreclosed upon is seen as a victim rather than a perpetrator, regardless of the actual circumstances of th default.

(Tomorrow: Risk and Consequences)

Introduction to Financial Crisis

In doing some research on the emerging trend of “strategic default” from an ethical perspective, I came across two programs, one audio, one video, on the banking crisis. While not strictly concerned with business ethics, these programs do a nice job in explaining the mistakes of human cognition/decision making that led to the crisis. Ultimately, I think, these perceptual shortcomings, combined with or magnified by unreflective profit-seeking, were to blame for the crisis, not a wholesale breakdown in ethics (or even a one-sided breakdown, which side you blame depending on your political/economic commitments).

For a very fun, non-technical introduction, you can’t do better than This American Life‘s episode, co-produced by the people responsible for the Planet Money blog, “The Giant Pool of Money“. It looks at the crisis from the perspectives of several players in the mortgage production chain (borrower, mortgage broker, banker) and describes how an excess of global capital helped drive the market frenzy for securitized mortgage obligations of various sorts. There is a follow-up program that is worth listening to, as well, “The Return to the Giant Pool of Money“.

For a more technical, yet still very accessible, introduction to the crisis, I suggest watching a brief talk given by Andrew Lo, who is the Director of the MIT Sloane School’s Laboratory for Financial Engineering. Titled Are Mathematical Models the Cause for Financial Crisis in the Global Economy?, he spends most of the talk providing the best introduction to the actual process of securitization that I’ve seen (and I’m someone who spent a good deal of my time actually working with the rating agencies, investment banks, and capital markets experts in putting these securities together). While I don’t think Lo actually answers the question adequately (the answer, in my opinion, is “no, the models didn’t cause the crisis, but the use of dodgy, short-term data and blind reliance on the models did”), the entire 50 minute talk is well worth your time if you want to get a high-level technical introduction to how securitization works and how it magnified the housing bubble and debt crisis.

I’m interested to hear from anyone who has other sources to help people understand the financial crisis. Part of what caused the bubble was that actors all along the mortgage production chain didn’t look carefully enough at the parts of the process they weren’t involved in. This led to a blind faith that “if the investment banks say the bonds are good, they must be good” and “if Countrywide thought this was a good loan, it must be a good loan”, etc. (I was actually guilty of the first one, looking back). If everyone had understood what other parts of the chain were doing, we might have seen the developing crisis much earlier than we did and been able to mitigate some of the worst effects.

Also, as I mentioned above, I’m working on an ethical analysis of “strategic default” and if anyone has any perspectives they’d like to share on it (pro, con, or otherwise), I’d be really interested to hear them.